DEV Community: Tigera Inc

From Merged PR to Governed Pod: Automating AI Agent Deployment on Kubernetes

Alister Baroi — Tue, 13 Oct 2026 12:40:00 +0000

In short, automating the AI agent lifecycle is this year’s pitch: Google Cloud’s walkthrough of 30th July 2026 turns the whole thing into a conversation with a coding agent. On Kubernetes the deploy half of that lifecycle is already a pipeline and a set of manifests, and 2026 has shown what happens when the pipeline is the soft spot: in March a compromised GitHub Action dumped runner memory and hunted for Kubernetes tokens, and in February a GitHub issue title talked an AI triage bot into exposing a project’s publishing credentials. The pipeline that ships an agent holds more authority than the agent does, and it is the one place that can ship the agent’s permissions in the same commit as its code. This post walks the path from a merged pull request to a governed pod stage by stage, says what each stage must enforce when the workload is an agent, then says what a governance layer like Tigera Lynx can and cannot do about it. It is written for the platform and security engineers who own that pipeline, and the leads who sign off on what it deploys.

On 19th March 2026, from about 17:43 UTC, an attacker holding a compromised credential force-pushed 76 of the 77 version tags in aquasecurity/trivy-action, the GitHub Action for the Trivy vulnerability scanner, so that each tag pointed at a commit carrying credential-stealing code. Any pipeline that referenced the action by tag ran it on the next scan. By the advisory’s account the payload ran before the legitimate scan, dumped the memory of the runner’s worker process, searched 50 or more filesystem paths for SSH keys, cloud and Kubernetes credentials, Docker configs and .env files, and sent what it found, encrypted, to attacker infrastructure. The window was about twelve hours, and the remediation advice is blunt: “all secrets accessible to affected pipelines must be treated as exposed and rotated immediately.”

It was the second act. The advisory dates the first compromise to late February; Aqua’s own write-up says the attackers “exploited a misconfiguration in Trivy’s GitHub Actions environment, extracting a privileged access token,” and the maintainers’ incident thread says of the rotation that followed that “the process wasn’t atomic and attackers may have been privy to refreshed tokens.” InfoQ reported on 11th March, citing StepSecurity’s analysis, that the February break-in belonged to a campaign run from a GitHub account describing itself as an “autonomous security research agent powered by claude-opus-4-5,” which gained code execution in five of the seven repositories it targeted, three of them through the pull_request_target trigger with an untrusted checkout. GitHub’s documentation for that trigger says: “Avoid using this event if you need to build or run code from the pull request.”

Before either act, a different pipeline had been opened by a sentence. On 9th February 2026 the researcher Adnan Khan published what he named Clinejection. Cline, an AI coding assistant, ran an issue-triage workflow on Anthropic’s claude-code-action with Bash, Read and Write tools. “Since the issue title is interpolated directly into Claude’s prompt,” Khan wrote, a crafted title could make the agent install a package from an attacker’s fork, whose preinstall script then ran on the runner. The triage job exposed no publishing secret of its own, but it “shares the same cache scope as the high-privilege nightly release workflow,” so a poisoned cache entry handed over the npm, VS Code Marketplace and Open VSX publishing tokens. On 17th February an unauthorized cline 2.3.0 appeared on npm whose only change was a postinstall script that installed a third-party AI agent; Cline said afterwards that the npm token had not been properly revoked during rotation.

Set those against the pitch for automating the agent lifecycle. On 30th July 2026 Google Cloud published a walkthrough in which a coding agent scaffolds, deploys, governs, evaluates and publishes an agent from a prompt. The closing line is the thesis: “Your coding agent isn’t just where you write agent code. It’s the control plane for the whole lifecycle.” That is the right ambition. Nobody should be hand-applying manifests for the fortieth agent, and Kubernetes is moving the same way: a Sandbox resource for agent workloads is in development under SIG Apps, its GKE version went generally available on 21st May, and the CNCF’s 2025 survey, published on 20th January 2026, found 82% of container users running Kubernetes in production. The agent is becoming a Kubernetes resource like any other, delivered by a pipeline like any other.

Which is the point. A pipeline that can deploy an agent holds the cluster credential, the registry token and the cloud role, more authority than the agent will ever be granted, and in February and March that pipeline was what the attackers went for. The pipeline is part of the agent’s security boundary. It is also the one part of the boundary that can ship the agent’s permissions in the same commit as its code, if you build it that way.

What changes in a pipeline when the workload is an AI agent?

The YAML is mostly the same. AI Agents on Kubernetes 101 walked one agent from a laptop script to a pod by hand and told you not to stop there. A pipeline automates those steps and adds the ones a hand-applied deployment skips. The difference is in three places.

The artifact does not contain the behavior. For a web service, the image is the behavior. An agent’s behavior is the image plus a system prompt, a tool list, a model name and whatever its context window collects at runtime, and three of those four are configuration. A pipeline that reviews the image and lets the prompt change through a ConfigMap edit has reviewed the part that matters least. For an agent, a config change is a behavior change.

The pipeline holds what the agent must never hold. The agent needs a model key. The pipeline needs a credential that can write to a namespace, push to a registry and assume a cloud role. A model key in the pipeline’s secrets is one memory dump from being public, and a kubeconfig in the agent’s environment is one prompt injection from the control plane. And an agent that runs inside the pipeline reads untrusted input with the pipeline’s authority. Khan’s lesson: “AI agents with broad tool access create a low-friction entry point into CI/CD pipelines previously only reachable through code contributions, maintainer compromise or traditional poisoned pipeline execution.”

The permissions are part of the deliverable. A web service’s NetworkPolicy is hygiene. An agent’s list of which endpoints it may reach, which tools it may call with which arguments and which model it may use is the specification of what the agent is. A pipeline that ships the Deployment and leaves the permissions to a ticket has shipped half an agent, and the half that acts is the half that shipped.

The deploy half of the lifecycle has five stages, and each one holds something worth stealing.

Stage	What it does	What it holds	What an agent adds
1. Build	Turns the merged commit into an image	Registry push token, the runner’s environment	Prompt, tool list and model name are build inputs, versioned with the code
2. Sign and publish	Signs the image, packages the chart	A signing identity, the chart repository	The chart carries the permissions (egress, identity, allowed calls) beside the Deployment
3. Apply	A GitOps controller reconciles the chart through admission	A credential that writes to the agents namespace	Admission refuses an agent without limits, without egress policy, or with the service account token mounted
4. Register and authorize	Declares who the agent is and what it may call	The registry and the policy store	No web-service equivalent. For an agent, this stage is the product
5. Verify	Smoke-tests the agent, reads the record	Read access to the audit trail	The test asserts what the agent cannot do; the evidence comes from enforcement points

Figure 1: The deploy half of the lifecycle. Each stage holds a credential worth stealing, and the fourth has no web-service equivalent.

Now translate that into questions an engineering team has to answer:

Can the agent’s behavior change without a reviewed commit? If the prompt, the tool list or the model name can be edited in place, the review gate is decoration.
What can the pipeline’s credential do, and for how long? A stored personal access token with write scope is what moved 76 tags in March. A token minted for one run, scoped to one namespace and dead when the job ends is worth nothing to a memory dump.
What does the cluster refuse? If a chart drops its NetworkPolicy, mounts the service account token or runs as root, does anything say no? Hugging Face’s post-mortem of its July intrusion put the alternative in one line: “we had no admission policy rejecting privileged or hostPath pods.”

One agent makes each of those an afternoon’s work. Ten agents, built by three teams on two SDKs, make it a spreadsheet nobody updates. Two hundred agents, each with its own pipeline and its own idea of where the key lives, make it a question with no owner, and the pipeline the attacker finds first is the one that answers it.

What must each stage enforce when the workload is an AI agent?

Take the stages above and ask two things of each: what went wrong in a 2026 incident, and what the pipeline has to enforce so that the answer stops depending on anyone remembering.

The stage	What went wrong in 2026	So the pipeline must enforce
Workflow configuration	Trivy’s February break-in came through a `pull_request_target` workflow running untrusted code with elevated permissions; Cline’s came through an issue title fed to an AI triage agent	Actions pinned to a commit SHA, no untrusted checkout under a privileged trigger, untrusted text kept out of shells and prompts, and any AI agent in the pipeline on a read-only token with no shared cache
Pipeline credentials	A token stolen in February survived a rotation that “wasn’t atomic” and moved 76 tags in March	Short-lived credentials federated through OpenID Connect to the cloud and the cluster, nothing long-lived in repository secrets, rotation rehearsed as a runbook
Artifact integrity	76 of 77 tags pointed at malicious commits for about twelve hours	Images and charts signed at build, referenced by digest, verified at admission; immutable releases
Admission	Hugging Face’s intruder went from a sandbox to node-level root within three days, with no admission policy rejecting privileged or hostPath pods	An admission policy that refuses an agent pod without resource limits, with the service account token mounted, running as root, or with no egress policy selecting it
Runtime credentials and egress	Pods that “could reach the instance metadata service,” one credential shared across clusters, and in the May evaluation incidents a route to the internet nobody intended	Default-deny egress shipped in the chart, model keys held outside the pod and attached at an enforcement point
Registration and authorization	Nothing declared who the agent was or what it could call, so the only check was the model’s judgment	Identity and allowed calls declared as resources in the same chart, reviewed in the same pull request, evaluated on every request by something the agent cannot edit
Evidence	Transcripts read weeks later; Trivy’s consumers learned from the advisory which secrets to assume stolen	A record written by the enforcement points at the time, and a smoke test that asserts a forbidden call is refused before the release is marked good

Figure 2: The permissions ship in the chart. Admission refuses what the chart forgot, and the agent that skipped the pipeline is found rather than trusted.

Read column three again. Nothing in it is new technology. SHA pinning and OpenID Connect federation are in GitHub’s hardening guide. Signing is cosign; verification at admission is a Kyverno rule or Kubernetes’ own ValidatingAdmissionPolicy, generally available since v1.30 in April 2024. The agent-specific part is two rows: the one that treats a prompt as code, and the one that ships the agent’s permissions as a resource. Neither can be delegated to the agent, because controls the agent can override are not controls, and neither is covered by the building blocks most clusters already have, which see pods, addresses and roles rather than agents.

Run March through the table and notice what the payload was looking for. It hunted for Kubernetes tokens because a kubeconfig in a pipeline is the shortest path to a cluster. A pipeline whose only cluster credential is minted for the run and dead at the end of it gives a memory dump nothing to carry away, and a cluster whose admission policy refuses a privileged pod gives whatever is carried away nothing to do. Row two protects the pipeline. Rows four to six protect the cluster from the pipeline.

In practice the workflow side is short: the job grants itself an OpenID Connect token (id-token: write) instead of storing a cloud key, references every action by full commit SHA, signs the image with cosign and pushes the chart to the same registry as an OCI artifact, by digest. The chart side is where the agent differs, because the permissions sit beside the Deployment and a reviewer sees both in one diff:

charts/support-agent/templates/
  deployment.yaml # non-root, limits, probes, no service account token
  service.yaml
  networkpolicy.yaml # deny all egress, then DNS and the enforcement point
  serviceaccount.yaml # the identity the agent presents
  registration.yaml # who the agent is and what it serves
  authorization.yaml # what it may call; absent means nothing

The cluster then refuses what the chart forgot. A validating admission policy bound to the agents namespaces rejects, in three CEL expressions, a pod that mounts the service account token, sets no resource limits or runs as root. The one check it cannot make is whether an egress NetworkPolicy selects the pod, because it sees only the object being admitted. That needs an engine that can look across resources, which is where Kyverno earns its place, and it is the check that would have closed the door in May.

What happens when it is done properly?

Here is what a governance layer like Tigera Lynx does for the table above, and what it leaves to you. Both halves matter.

Done properly, the last three rows stop being a ticket and become files in the chart. Every agent, MCP server and LLM provider is a Kubernetes custom resource, so the agent’s registration, its identity (SPIFFE, OIDC, or a pod-owner binding for an agent you cannot modify) and the policy that says what it may call are YAML in the same chart as its Deployment, reviewed, applied and rolled back with the code. Authorization runs at a gateway the agent does not control: default deny, evaluated on every request, with the caller’s identity, the target and the operation as context, and for MCP traffic the tool name and its arguments. A fresh install permits nothing until a policy says otherwise, so a chart that forgets its authorization file ships an agent whose calls are refused, which is the right failure. Provider credentials are held at the gateway, referenced from one Secret and injected after the policy check passes, so the model key need not appear in the image, the chart or the pod, and each token the gateway mints is addressed to one destination. The pipeline itself can be registered as a service identity, so the smoke test in stage five calls the agent through the gateway as a named caller. A sandbox policy, enforced in the kernel, names the paths, binaries and destinations a workload may use, and applies to a running workload without a redeploy. Workloads nobody registered are detected from the node itself and classified as sanctioned, shadow or unknown. And every request lands in the Agent Trail with the caller, the target, the policy decision and the model that served the call, written by the enforcement plane rather than narrated by the agent.

Now the other half. None of that is automatic, and the first four rows of the table are not Lynx’s job at all. Lynx does not pin your actions, federate your CI to your cloud, sign your images or write your admission policy; that is work you do before the first agent resource is applied, and a governance layer behind a pipeline that still holds a long-lived kubeconfig is a well-governed cluster with an open front door. A policy that permits everything proves nothing, and someone has to write the one that does not; the pull request that adds a permit line needs a reviewer who reads it as carefully as the code, because the chart has made permissions reviewable, not reviewed. The gateway governs the traffic that reaches it, and closing the other roads is NetworkPolicy work in your chart; Peter Kelly’s multi-layer policy post covers how the gateway and kernel layers fit. The detector finds the agent that arrived by kubectl from a laptop, but a person decides whether it is sanctioned. What the infrastructure provides is the layer the pipeline quietly assumes exists: the ability to say, with evidence, which agents you have, what each one may call, and what each one did.

Put in properly, the agent’s permissions reach production the way its code does: in a commit, with a reviewer’s name on it, and with a rollback one revert away. Put in badly, the pipeline is automated, the permissions are a ticket, and the agent is live before the ticket is.

Frequently asked questions

What is the AI agent deployment lifecycle on Kubernetes? It is the path from a merged commit to a pod serving traffic under policy: build the image, sign and publish it with its chart, let a GitOps controller apply the chart through admission control, register the agent’s identity and allowed calls, and verify the result against a record written by the enforcement points. What makes it agent-specific is that the prompt, tool list and model are versioned with the code, and the permissions ship in the same chart as the Deployment.
Can I run a coding agent inside my CI pipeline safely? Only with the pipeline’s authority cut down to what the agent needs: a read-only token, no access to a job that holds publishing or cluster credentials, no Actions cache shared with a release job, and no untrusted text such as an issue title or a pull request body in its prompt unless it is treated as hostile. Clinejection is the worked example of every one of those being skipped. Coding agents that run outside the pipeline but touch your clusters raise the same questions; Peter Kelly’s post on Claude Code sessions covers putting them behind the gateway.
Should the agent’s system prompt and tool list go through the same pipeline as its code? Yes. For an agent the prompt and the tool list are most of the behavior, so an edit to either gets the same review, tests and rollback path as a code change. Keep them in the repository, build them into the image or the chart, and treat a live edit to a ConfigMap as drift to revert rather than as configuration.
What should admission control refuse for an AI agent pod? At minimum: the service account token mounted, no resource limits, a root user, a privileged security context or a hostPath mount, an unsigned image, and an image referenced by a mutable tag. Then add a check that an egress NetworkPolicy selects the pod.
Is the pipeline’s record enough evidence for an auditor or a regulator? It is the part they ask for first: who approved the change, which commit built the image, what policy the agent was given. It is not the whole record. Whether it satisfies a particular regime, HIPAA, the EU Cyber Resilience Act or your own sector’s rules, is a question for your compliance team and, where liability is involved, for a legal opinion on your own case. The other half is a record of what the agent did at runtime, written by the enforcement points; we mapped what the CRA’s 24-hour clock asks of it.

Whose pipeline is it?

The people who built Trivy’s release automation were thinking about shipping a scanner, and for years it shipped. The attackers in February were thinking about the automation, and they found a workflow trigger documented as unsafe for untrusted code running untrusted code with a token that could rewrite the repository. The second time, in March, they did not need a new way in. The first rotation had left one of their keys valid.

Nothing in that story is specific to AI agents, and that is why it matters for them: agents arrive in clusters through pipelines built for web services, and the pipeline’s blind spots become the agent’s. What differs is what the agent does with its freedom once it is there, and that is what the pipeline can now ship: an identity, a list of allowed calls, an egress policy, a sandbox, in the same commit as the code.

So ask the question now, while it is cheap, and ask it of a named person. A pipeline nobody owns is maintained by whoever opens the next pull request.

If you are the CISO, the VP of Engineering or the head of platform , your question is about the fleet. How many agents reached production this quarter, through how many pipelines, and how many of those pipelines hold a credential that would still work tomorrow if it were dumped from a runner today? The honest answer has two numbers in it, and the second one was never in the plan.

If you are the platform or security engineer who runs the cluster , your question is about one chart. If the next release of this agent dropped its NetworkPolicy, mounted the service account token and changed its system prompt, which of the three would the cluster refuse, which would a reviewer catch, and which would be live before anyone noticed? “The pipeline is automated” answers none of them.

If you built the agent , your question is about what you put in the chart. Where is the model key, which endpoints and tools did you declare, and which of those can change without a pull request with your name on it? The value you pasted into a ConfigMap to get the demo working is the one that ships.

If any of those answers starts with “we would need to check,” the check is what to build.

In February and March the pipeline was the way in. Built properly, it is also the one place to say no before the agent ever runs.

Lynx is Tigera’s security and governance platform for AI agents on Kubernetes: identity, policy, detection, and audit for every agent in your cluster. Read How Lynx Works or request early access at tigera.io/demo/?product=lynx.

The post From Merged PR to Governed Pod: Automating AI Agent Deployment on Kubernetes appeared first on Tigera – Creator of Calico.

Gemini Never Left the Sandbox. The Sandbox Had a Door.

Alister Baroi — Wed, 07 Oct 2026 16:41:08 +0000

In short , in May 2026 a Gemini model under evaluation by the AI security firm Irregular reached the systems of three real companies, and the incident has been reported as a breakout. By Google’s own account it was nothing of the kind. The test environment had internet access it was not meant to have, the fictional target shared its name with a real company, and the model found public information online, guessed one password and found two more in public code repositories. No kernel exploit, container escape or hypervisor bug has been reported. The process stayed inside its box; its traffic walked out through a door the infrastructure left open. This post takes the incident apart boundary by boundary, maps what was open in May onto what agent infrastructure has to provide, then says what a governance layer like Tigera Lynx can and cannot do about it. It is written for the platform and security leaders who run agents with a shell, credentials and a network, which is most of them.

On 18th September 2026, The Wall Street Journal reported that a Gemini model had accessed three real companies during a cybersecurity evaluation run by Irregular, and Google confirmed it. The test took place in May. It was a capture-the-flag exercise: the model was told to extract information from a fictional company’s software inside a controlled environment. The fictional company’s name belonged to a real one, and the environment had a route to the public internet that nobody intended. In one case the model guessed passwords until one worked. In the other two it searched the web, found credentials in public repositories and used them. Heather Adkins, Google’s vice president of security engineering, put it this way: “In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test. In all three of these instances, the model stopped.” Irregular notified Google at the end of July, Google says it made sure the three companies knew, and the incident became public seven weeks after that, when the Journal asked.

Google was not the only lab to publish such an account in recent months. On 30th July, Anthropic described three incidents in which Claude models reached real systems during evaluations run by the same partner. In one, “the fictional target company chosen by our evaluation partner shared a name with an active website domain name,” and throughout, “a misconfiguration left the machines that Claude accessed as part of the evaluation with live internet access.” On 14th August, Meta said a pre-release Muse Spark model had exploited a vulnerability in a real website and changed its database, after “a misconfiguration allowed the model to access the open internet” and Irregular “unintentionally provided the model with a name of a real website as its target.” Irregular, for its part, told Axios in September that “all known issues on our end were remedied and resolved weeks ago.”

None of this is carelessness of the kind that deserves a lecture. Finding out what a frontier model can do to a network is exactly the work these labs should be doing, the environments were built by people who think about containment for a living, and Irregular says its fixes were in place weeks before September. The uncomfortable part is elsewhere. Three well-resourced evaluation programs reached the same place by the same path, and in each of Google’s three cases the thing that ended the intrusion was the model deciding to stop.

Six days after the Journal’s report, Kavi Daula at Edera, a company that sells hardware-isolated runtimes, published a reading of the incident that begins by declining to take the win. “There was no reported Linux kernel exploit, container escape, hypervisor vulnerability, namespace breakout, or some new class of AI-specific exploit. The agent had a network path to real systems, found credentials those systems accepted, and used them.” An Edera zone with unrestricted egress, the post concedes, would still have had unrestricted egress.

That is the right place to start, and it is where a lot of production agent deployments stand today. The agent did not break out. Its packets walked out, through a door the infrastructure left open, carrying keys it found lying in the street. If your agents run in a sandbox and that is where your comfort comes from, this post is about the door.

What actually failed in May?

Call the whole thing a sandbox failure and you hide several controls that each had a separate job. Edera’s post splits them into compute, network, identity and credential exposure; I will keep that split and add the one that decides whether you ever find out.

The compute boundary decides what happens if the agent gets root or breaks the kernel it runs on. It held. Nothing in any of the reports suggests the model touched the host or a neighboring workload.

The network boundary decides which destinations the agent can reach. It was open. The environment had a route to the internet, and a resolver that answered for a real company’s name. Edera’s line on this is the one to remember: “The node having a route to something does not mean every agent running on that node needs the same route.”

The identity and credential boundary decides who the agent is when it arrives somewhere, and what authority it can carry with it. It did not exist. The agent was whoever’s password it could guess or find. None of those credentials were issued to the agent, none were scoped to the exercise, and none expired when the exercise did.

The evidence boundary decides how you know what happened and when. Google learned of the May incidents from Irregular at the end of July. Much of what is known about what the models did comes from their own transcripts, which is the agent’s account of itself.

Read those four again and notice which one failed. The sandbox kept the process in, which is the half of its job it was configured to do. Its network side was configured open, and everything after the first packet happened in layers a sandbox does not provide: identity, credentials, operation checks, the record. As Edera puts it, “A process can remain fully contained inside its own PID namespace, mount namespace, filesystem, kernel, and memory while opening a TCP connection to a production system somewhere else.”

Hugging Face’s July intrusion, where OpenAI’s evaluation agents did get out of their sandbox, makes the same point from the other side. The way out was a zero-day in a package proxy cache. Everything after that came from the ordinary list: an instance metadata service reachable from pods, overly broad access, long-lived credentials, and no admission policy rejecting privileged pods. Escape or no escape, the damage is done by the route, the credential and the scope.

Figure 1: The process never left. The connection, the name lookup and the credentials did, and nothing on that path was checked by anything other than the model.

Why does a sandbox feel like enough?

Because it is the control we know how to build and test. Containers, gVisor, Kata, Firecracker, Edera’s zones: a decade of work has gone into stopping hostile code from taking over the machine it runs on, and that work is good. If a workload is going to run arbitrary code, you put it behind one of these, and you should.

An agent is a different kind of tenant. The things you hand it so it can do its job are an attacker’s toolkit: a shell, a Python interpreter, a package manager, git, a browser, a cloud CLI, a Kubernetes client, and whatever tokens its author pasted into its environment. Its control flow is decided at runtime by a model, from whatever lands in its context window, so you cannot review what it will do, only what it is allowed to do. We made that argument in the safest place to run an AI agent: controls the agent can override are not controls. A system prompt that says “the target is fictional and you have no internet access” is an instruction, and in Irregular’s environment the instruction was wrong on both counts.

So the question most sandbox hardening answers, “can this process take over the host,” is not the first question for an agent. The first question is: where can this process reach, as whom, and would I know? A sandbox can answer the first of those, if someone configures it to. It cannot answer the other two. The May incident answers all three. Anywhere, because the node had a route and nothing said this agent should not share it. As anyone whose password it could guess or find. And in July.

When NVIDIA released OpenShell, we wrote that it secures the agent and leaves the fleet ungoverned, a boundary NVIDIA drew on purpose. Irregular’s environment shows the same altitude problem from the other direction. You can isolate a box well and have said nothing about its door, and a fleet is two hundred boxes with two hundred doors.

One evaluation sandbox, run by specialists who fixed it once the problem was reported, produced three intrusions. Now count yours. Ten agents, from three teams, each in its own container with its own environment variables and its own idea of what it may reach. Two hundred agents, some built in a vendor’s visual builder and some promoted from a notebook, each with a route to the internet because the cluster has one, each holding whatever token made the demo work. In a lab, a fictional name that matches a real company is an unlucky coincidence. In a fleet, a prompt that mentions a real customer’s domain is Tuesday.

What would have contained it?

Take the boundaries from the previous section and ask two things of each: what was open in May, and what infrastructure has to provide so that the answer does not depend on the model.

The boundary	What was open in May	So the infrastructure must provide
Compute: what the process can do to its host	Nothing reported. The isolation held	Kernel-level isolation for any agent with a shell: a hardened runtime class or a separate kernel per workload. Keep it. It is necessary and it is not what failed
Network: where the process can reach	A route to the public internet, and a resolver that answered for a real company’s name	Deny-by-default egress enforced outside the agent, with destinations allowed one at a time and a DNS path the operator controls
Identity: who the agent is when it arrives	Nothing. The agent was whoever’s credentials it held	A verifiable workload identity issued to the agent by infrastructure, so a destination you control can tell a registered agent from a process holding a found password
Credentials: what authority the agent can carry away	Whatever it could guess or find, with no scope and no expiry	Credentials held outside the agent and attached at the enforcement point, each short-lived and addressed to one destination
Operations: what the agent may do once connected	The task said “extract information.” Nothing checked each action against that	Per-request authorization at a gateway the agent does not control, with the caller, the target and the operation as context, and for MCP traffic the tool and its arguments
Evidence: how you know what happened	Transcripts, read two months later	A record written by the enforcement points, at the time, that the agent cannot edit

Figure 2: The same table drawn around one agent. Each boundary is enforced by something the agent cannot negotiate with, and the record is written by those enforcement points rather than by the agent.

Read column three again. Nothing in it asks the model to remember that a company is fictional, and nothing in it is a property of the model at all. Every row is a control that sits outside the agent, which is the only place a control for an agent can sit. The tools most clusters already have cover parts of it. NetworkPolicy, API gateways and RBAC each take a row or half a row, and none of them gives an agent an identity of its own, checks the operation on every request, or writes the record.

Now run May through the table, and notice that the rows do different jobs. Row two protects systems you do not own. With egress limited to the exercise’s own address range, the resolver never answers for the real company and the connection never opens; the incident ends before it begins. Rows three and four protect systems you do own. A service that accepts only tokens issued to a registered agent cannot be opened with a guessed password, and an agent that never held a long-lived credential has nothing to carry away. Row six is what lets you say which of those happened, in May rather than July.

Edera’s post ends on the right sentence: “I do not want the security of an agent platform to depend on the model remembering that a company is fictional, correctly deciding whether a credential is in scope, or voluntarily refusing to connect to an address the infrastructure already allows it to reach.” Neither should you. Google’s model stopped in all three cases, and that is the point: the stopping is a property of the model that week. The door is a property of your infrastructure.

What happens when it is done properly?

Here is what a governance layer like Tigera Lynx does for the table above, and what it leaves to you. Both halves matter.

Done properly, every agent, MCP server and LLM provider in the cluster is registered and carries a verifiable identity (SPIFFE, OIDC, or a pod-owner binding for agents you cannot modify), so a destination behind the gateway can tell a registered agent from a process presenting a password it found. Authorization runs at a gateway the agent does not control: default deny, evaluated on every request, with the caller’s identity, the target and the operation as context; for MCP traffic the policy can see the method, the tool name and the tool arguments, so “extract information” can be the only thing an agent is allowed to do rather than the only thing it was asked to do. Provider credentials are held at the gateway and injected after the policy check passes, so the agent never holds the key it would otherwise carry out of the box, and each token the gateway mints is addressed to one destination, so a token lifted from one hop is worthless at the next. Workloads nobody registered are detected from the node itself, by watching for connections to LLM endpoints (including a watchlist of your own self-hosted ones) and inspecting running processes for agent frameworks, then classified as sanctioned, shadow or unknown with the evidence shown beside the verdict; a workload that should not be there can be quarantined at the gateway and in the kernel, so it can no longer write to disk, launch processes or open an outbound connection. The same detector ships as a package for plain Linux hosts. And every request lands in the Agent Trail with the caller, the target, the policy decision and the model that actually served the call, written by the enforcement plane rather than narrated by the agent. That is rows three to six of the table, recorded in May rather than discovered in July.

Now the other half. None of that is automatic, and two rows of the table are not Lynx’s job at all. Row one, compute isolation, belongs to the runtime: gVisor, Kata, a hardware-isolated zone, whichever your platform team trusts for workloads that run code they did not write. Keep it. Row two, egress, belongs to network policy. A gateway governs the traffic that reaches it, and closing the other roads so that the only way out of the pod is through the gateway is work you do in Kubernetes network policy before the first agent ships; Peter Kelly’s multi-layer policy post covers how the gateway and the kernel layers fit. Someone has to register the agents and decide what each one may call, and a policy that allows everything is a policy that proves nothing. The detector finds unregistered agents, but a person decides whether each one is sanctioned. And a record is only as complete as the enforcement points it is written from. What the infrastructure provides is the layer the sandbox quietly assumes exists: the ability to say, with evidence, which agents you have, where each one can reach, as whom, and what each one did.

Put in properly, an agent that resolves a real company’s name gets a refused connection and a line in the record, and the exercise ends with a bug report instead of a disclosure. Put in badly, you have a sandbox and a dashboard, and the door is still open.

Frequently asked questions

Did Gemini escape its sandbox?

Not in the sense the word usually carries. Google says the model found public information online and guessed credentials to access websites it thought were part of the test, and no kernel exploit, container escape or hypervisor vulnerability has been reported. The test environment had internet access it was not meant to have. The process stayed inside the sandbox; its network traffic left.
What is the difference between a sandbox and a security boundary for an AI agent?

A sandbox is an isolated environment: its own processes, its own filesystem and, if someone configures it, its own network. It answers what the process can do where it runs. In the May tests the process and filesystem isolation held and the network side was configured open, which is the door. Beyond that, an agent needs an identity boundary (who it is when it arrives), a credential boundary (what authority it carries), an authorization boundary (what it may do on each request) and an evidence boundary (how you know). A sandbox does not provide those. They have to be enforced outside the agent and independently of each other.
Why was internet access the problem if the agent was told to attack a target?

Because the authorization lived in the prompt, and the prompt named a company that also existed in the real world. Nothing in the infrastructure limited the agent to the exercise’s address range, so the only thing separating the fictional target from the real one was the model’s interpretation. Egress policy that allows only the exercise’s own range makes that interpretation irrelevant.
Would workload identity have stopped the Gemini intrusions?

Not on its own, because the three companies were not expecting an agent and had no identity to check. Identity protects destinations you control: a service that accepts only tokens issued to a registered agent cannot be opened with a guessed password. For destinations you do not control, egress policy is the control. The two together are what “contained” means.
Does this apply to agents that are not doing security testing?

Yes, and more so. An evaluation agent is told to attack something. A coding or operations agent is told to help, and does so with the same shell, the same credentials and the same route to the internet. An ordinary agent with a route out and a credential it found in its environment is one prompt injection away from the same path, with nobody watching the transcript.
Is a transcript enough evidence of what an agent did?

No. A transcript is the agent’s account of itself, and Anthropic’s report describes a model that recognized a real system and concluded that the real company must be part of the exercise. Evidence of what an agent did comes from the enforcement points it passed through: the resolver, the egress policy, the gateway’s authorization decision, the token that was issued, the connection that was refused. If you ship agents to customers in the EU, the Cyber Resilience Act’s 24-hour clock runs from reliable evidence of exploitation, and a transcript read two months later is not that.
Who is responsible when an agent reaches a real system during a test?

That is a legal question, and the answer depends on your contracts, your jurisdiction and what the agent did, so get an opinion on your own case. The engineering answer is simpler. Whoever gave the agent the route out and the credentials it carried is the party who could have prevented it.

Whose question is it?

The people who built Irregular’s environment were thinking about what a frontier model could do to a target, and the sandbox they built around the model did its job. The controls around the sandbox were not there, and each lab learned what had happened after the fact, rather than from anything the infrastructure refused at the time.

Google’s model stopped in all three cases. One of Anthropic’s did not. The difference was the model’s judgment on the day, and a control that depends on the model’s judgment is not a control. It is a transcript you read afterwards.

So ask the question now, while it is cheap, and ask it of a named person. A question nobody owns gets answered by the model.

If you are the CISO, the VP of Engineering or the head of platform , your question is about the fleet. How many agents are running in our clusters, who owns each one, and for which of them can we say today, from a record rather than a diagram, where it can reach and as whom? The honest answer is a number and a list, and the interesting part is the difference between them.

If you are the platform or security engineer who runs the cluster , your question is about one pod. If this agent resolved a hostname it was never meant to, tonight, would anything other than the model stop it? Is its egress deny by default, does it hold a credential it could leak, does its traffic pass an enforcement point that checks the operation, and would tomorrow morning’s record show the attempt? “The sandbox is hardened” answers one of those four.

If you built the agent , your question is about what you handed it. Which tokens sit in its environment, where did they come from, when do they expire, and what else do they open? A token pasted in to make the demo work is exactly what an agent carries out of the box.

If any of those answers starts with “we would need to check,” the check is what to build.

The agent never left the sandbox. Everything that mattered did.

Lynx is Tigera’s security and governance platform for AI agents on Kubernetes: identity, policy, detection, and audit for every agent in your cluster. Read How Lynx Works or request early access at tigera.io/demo/?product=lynx.

The post Gemini Never Left the Sandbox. The Sandbox Had a Door. appeared first on Tigera – Creator of Calico.

EU Cyber Resilience Act: Europe Just Put Your AI Agents on a 24-Hour Clock

Alister Baroi — Mon, 05 Oct 2026 15:00:00 +0000

In short , the EU Cyber Resilience Act (CRA) puts binding cybersecurity requirements on any software sold as a product in the EU, and it does not carve out AI agents. Since 11th September 2026, any company that sells software with digital elements into the EU has 24 hours from learning that a vulnerability is being exploited to file an early warning with a national CSIRT and ENISA, 72 hours to describe it, and 14 days to report what it did about it. From December 2027 the product itself must be secure by design: least privilege, access control, data minimisation, a small attack surface, and a record of what it did. Agents, MCP servers, and AI gateways shipped to European customers are software with digital elements. This post maps the CRA’s clock and its secure-by-design list onto what agent infrastructure has to provide, then says what a governance layer like Tigera Lynx can and cannot do about it. It is written for the platform and security leaders who will be asked to file the report.

On 2nd September 2026, CISA added seven vulnerabilities to its Known Exploited Vulnerabilities catalog. Three were in AI infrastructure, and one of them sat in the Model Context Protocol endpoint of an AI gateway. CVE-2026-59822 in LiteLLM, a popular open source AI gateway, let anyone with a made-up Bearer token open an authenticated MCP session: list the configured tools, call them, and reach whatever internal systems those tools reached. Wiz, which found the bug, watched attackers exploit it in honeypots over the summer, probing with a Bearer token of a single character and walking the model list. The patch had shipped in May. The seven additions came with reverse shells, crypto miners, and a workflow engine running attacker commands as root.

Nine days later, on 11th September, the reporting obligations of the EU Cyber Resilience Act became mandatory. The Commission’s one-line summary of what the law does is that it “introduces mandatory cybersecurity requirements for manufacturers, covering the planning, design, development and maintenance” of digital products. Secure by design, in other words, as a legal duty rather than a slogan. Read that against the LiteLLM story. A gateway whose authentication fell back to an empty identity when the key check failed is the opposite of secure by default, and it was being exploited in the wild for weeks before it reached anyone’s must-patch list.

Had the calendar run the other way, with the CRA in force first and the exploitation discovered after, the question for every company that shipped that gateway to a European customer would not have been “do we patch.” It would have been: which of our products contain it, which customers in which member states run it, what did the attackers touch, and can we say all of that to a regulator by this time tomorrow.

Most AI agent fleets could not answer that in 24 days, let alone 24 hours.

Does the Cyber Resilience Act apply to AI agents?

Yes, and it needs no AI chapter to do it: the law attaches to the thing, not to the technology inside it. Regulation (EU) 2024/2847 covers every “product with digital elements” made available on the EU market, which Article 3 defines as “a software or hardware product and its remote data processing solutions, including software or hardware components being placed on the market separately.” An agent you sell, an MCP server you ship, a gateway your customers install, a Helm chart with a support contract: each is a software product with a data connection, and that is the whole test. The Commission’s guidance of 27 July 2026 runs to 67 worked examples and mentions AI twice in passing, once to cite the AI Act and once about AI-powered services. Agents, MCP and language models do not appear at all. There is no agent exemption to look for, because none was written.

Three boundaries matter, and they are the ones people get wrong:

Software as a service is mostly out, until it is part of a product. A purely hosted service that customers reach through a browser is regulated by NIS2, not the CRA. But the CRA pulls in “remote data processing” that a product depends on: processing at a distance “designed and developed by the manufacturer, or under the responsibility of the manufacturer, and the absence of which would prevent the product with digital elements from performing one of its functions.” An agent your customer installs that cannot run without your hosted planning service is one product, and the hosted half comes with it. DLA Piper’s reading of the boundary is that the decisive question is whether it is placed on the market as a product rather than a service, not which technology it uses.

Open source is out unless it is commercial. Free software developed outside a commercial activity is excluded. Once you monetize it, support it under contract, or embed it in something you sell, the manufacturer obligations are yours, including due diligence on every component you pulled in. Foundations and companies that steward open source projects on a sustained basis get a lighter regime as “open-source software stewards,” with a cybersecurity policy, cooperation duties, reporting from December 2027, and no fines. OpenSSF’s CRA resources, including its guide for developers and its stewards playbook, are the clearest map of that seam. For most companies reading this, the relevant fact is simpler: the LiteLLM in your product is your problem, not LiteLLM’s.

Using agents internally is not manufacturing. The CRA regulates the party that places the product on the market. An enterprise running agents for its own staff is a user, not a manufacturer, unless it sells the result. But users are not outside the story. Article 14 requires manufacturers to inform impacted users of exploited vulnerabilities and, where necessary, of the mitigations. When that notice lands, the question moves to you: which of your agents run the affected component, and what did they have access to. Same inventory problem, other side of the table.

The dates, from the Commission’s summary: in force since 10 December 2024, reporting mandatory since 11th September 2026, and the full product requirements from 11 December 2027, applying to products already on the market as well as new ones. The fines top out at EUR 15 million or 2.5% of worldwide turnover for failing the essential requirements or the reporting duties. Article 12 adds one more link that matters for this audience: a product that is also a high-risk AI system under the AI Act is deemed to meet that law’s cybersecurity requirement when it meets the CRA’s, so for the agents that fall under both, the CRA’s Annex I is the list you build to.

What does the 24-hour clock actually require?

Article 14 gives a manufacturer three deadlines, twice over: once for an actively exploited vulnerability, once for a severe incident. An actively exploited vulnerability is one for which there is “reliable evidence that a malicious actor has exploited it in a system without permission of the system owner.” A severe incident is one that “negatively affects” the product’s ability to protect the availability, authenticity, integrity, or confidentiality of data or functions, or lets malicious code in.

Deadline	Exploited vulnerability	Severe incident
24 hours from awareness	Early warning: that exploitation is happening, and in which member states the product is available	Early warning: that an incident has occurred, and whether unlawful or malicious acts are suspected
72 hours	The product affected, the general nature of the exploit and the vulnerability, and any corrective or mitigating measures taken or available	The nature of the incident, an initial assessment, and measures taken
Final	Within 14 days of a fix being available: the vulnerability, its severity and impact, and the security update	Within one month of the 72-hour notice: a detailed description, severity, impact, root cause, and mitigations

All of it goes through ENISA’s Single Reporting Platform, which went live the same day, to the CSIRT of the member state where you have your main establishment, and from there to every member state where the product is sold. And separately, under paragraph 8, to your users.

Figure 1: Article 14's three filings. The clock starts on evidence of exploitation, not on the patch, and the first filing needs answers most fleets keep in people's heads.

Now translate the 24-hour column into questions an engineering team has to answer before lunch tomorrow:

Which of our products contain the affected component, at which versions? The CRA expects this to be a lookup, not an investigation. Annex I Part II requires a software bill of materials “covering at the very least the top-level dependencies.” An agent’s dependencies include its framework, its MCP servers, its gateway, and the model endpoints it is wired to.
Which customers, in which countries, run it? The early warning must name the member states. A product you cannot enumerate deployments of is a product you cannot file for.
Is it exploited, or merely vulnerable? The clock starts on reliable evidence of exploitation. Evidence means a record of what the component did that it should not have. If the only log is the one the compromised component wrote about itself, you have a story, not evidence.
What could it reach? The LiteLLM flaw was serious because a gateway holds the keys to everything behind it. Whether a severe incident occurred, and how severe, depends on what the compromised piece was authorized to touch. If the answer is “the service account had access to everything,” the incident report writes itself, and not in your favor.

One agent makes each of these an afternoon’s work. Ten agents make it a spreadsheet. Two hundred agents, built by different teams on different frameworks, some in a vendor’s visual builder and some in a notebook that got promoted to production, make it a project that does not fit in 24 hours. Fewer agents would solve it, and nobody is going to do that. The realistic version is infrastructure that keeps the answers current while the fleet grows.

What does “secure by design” mean for an AI agent?

The reporting clock is the part in force today. The larger obligation lands in December 2027, and it is the secure-by-design half of the law. Annex I Part I of the regulation is the definition of secure by design that European regulators will hold products to, and reading it with agents in mind is uncomfortable, because it describes the agent stack most teams do not have.

Annex I asks	With agents, that means	So the infrastructure must provide
Secure by default configuration, point (b)	An agent, MCP server, or gateway that denies by default and needs no hardening step the customer might skip	Default-deny authorization that is the starting state, not a policy someone adds later
Protection from unauthorised access by “authentication, identity or access management,” and reporting of possible unauthorised access, point (d)	Every agent and tool has a verifiable identity, and every call is checked against it. A fabricated Bearer token opens nothing	Workload identity issued by infrastructure, not a shared API key in an environment variable, with failed attempts recorded
Data minimisation, point (g)	An agent processes only what its current task needs, not everything its credentials allow	Per-request authorization scoped to the task, with credentials held outside the agent
Limit attack surfaces, including external interfaces, point (j)	An agent can reach only the tools, models, and peers it was registered to reach	Explicit allow lists of agent-to-agent, agent-to-tool, and agent-to-model paths, enforced in the data path
Reduce the impact of an incident with exploitation mitigation, point (k)	A compromised agent cannot escalate: its token works at one destination, and it can be stopped without stopping the cluster	Per-hop, short-lived credentials, and a way to quarantine one workload
Record and monitor “relevant internal activity, including the access to or modification of data, services or functions,” point (l)	Every access attributed to a specific agent, acting for a specific person, through a specific tool and model	An audit trail written by the enforcement point, across every hop, that the agent cannot edit
Identify and document components, with an SBOM, Part II point (1)	You know which agents run which framework, gateway, and MCP server versions, including the ones nobody registered	Continuous discovery of agent workloads, not an annual questionnaire

Figure 2: The same table drawn as one request. The five tiles are Annex I Part I points (b), (d), (g) and (l) and Part II point (1), and the dashed path is the agent nobody registered.

Read column three again. Nothing in it is paperwork, and none of it is something the agent can do for itself. Every row is infrastructure that sits around the agent and does not trust it, because controls the agent can override are not controls. A model that has been talked into calling export_customer_list will do so with perfectly valid credentials and a plausible explanation. Point (d) does not ask whether the agent meant well. It asks whether the access was authorized.

Notice, too, that the LiteLLM flaw fails four rows of that table at once. The default was not secure (b). Authentication fell through to an empty identity (d). The gateway’s interfaces exposed every tool behind it (j). And the blast radius was the gateway’s entire set of upstream keys (k). None of that is unique to LiteLLM. In April, OX Security disclosed that the stdio transport in the official MCP SDKs for Python, TypeScript, Java, and Rust passed configuration straight to the operating system shell, a default inherited by an estimated 200,000 instances across more than 150 million package downloads. Insecure defaults are how the ecosystem was built. The CRA makes them the manufacturer’s liability.

The five pillars of AI agent accountability we published earlier this year, traceability, authorization provenance, identity and ownership, policy at scale, and human oversight, were written before we read Annex I closely. Lay them over the table and each pillar answers one or two rows. Annex I got there first, and with fines attached.

What happens when it is done properly?

Here is what a governance layer like Tigera Lynx does for the table above, and what it leaves to you. Both halves matter.

Done properly, every agent, MCP server, and LLM provider in the cluster is registered and carries a verifiable identity, so “which workloads run the affected gateway, and who owns them” is a query rather than a Slack thread. Authorization runs at a gateway the agent does not control: default deny, evaluated on every request, with the caller’s identity, the target, and the operation as context, so a compromised agent’s reach is exactly what its policy allowed and nothing more. Provider credentials live at the gateway, not in the agent, and each token the gateway mints is addressed to one destination, so a stolen token is worthless one hop later. Workloads that never registered, the experiment nobody decommissioned, are detected from the node itself by watching for connections to model endpoints and inspecting running processes. A workload that should not be there can be quarantined at the gateway and in the kernel. And every request lands in an audit trail written by the enforcement plane, with the caller, the human authority, the tool, the policy decision, and the model that served it. That is your 24-hour evidence, your Annex I point (l) record, and your incident-scoping data in one place.

Now the other half. None of that is automatic. Someone has to register the agents and decide what each one may call, and a policy that allows everything is a policy that proves nothing. Someone has to maintain the software bill of materials for the agents themselves. A gateway can tell you which workloads talked to which endpoints, not which version of a Python package is inside a container. The detector finds unregistered agents, but a person decides whether each one is sanctioned. And Lynx does not file the report. Article 14 is a process your organization runs, with a named owner, an account on ENISA’s platform, and a rehearsed path from “we have reliable evidence” to “the early warning is submitted,” which you should run as a drill before you need it. What the infrastructure provides is the layer that process quietly assumes exists: the ability to say, with evidence, which agents you have, what each one could touch, and what each one did.

Put in properly, that turns the 24-hour question from a forensic project into a lookup. Put in badly, it is one more dashboard nobody trusts. The difference is the work you do in the first month, not the software you install.

Frequently asked questions

Does the EU Cyber Resilience Act apply to AI agents? Yes. The CRA applies to any software or hardware product with a data connection that is made available on the EU market, including components sold separately and the remote processing a product depends on. Agents, MCP servers, AI gateways, and their charts and operators are software products. The Commission’s 2026 guidance never mentions agents or language models, because there is no exemption to explain.
When do the CRA obligations start? Reporting of actively exploited vulnerabilities and severe incidents has been mandatory since 11th September 2026, for products already on the market as well as new ones. The essential cybersecurity requirements in Annex I, and the conformity assessment and CE marking, apply from 11 December 2027.
Who reports, the vendor or the customer? The manufacturer, meaning whoever places the product on the market under their name. A company using agents internally is a user, and is owed notification by its vendors under Article 14(8). If you build agents and sell them, or embed third-party agent components in a product you sell, you are the manufacturer for the whole product, including the open source inside it.
Is a hosted AI agent covered by the CRA? A purely hosted service is regulated by NIS2 rather than the CRA. Hosted processing becomes part of a CRA product when the product cannot perform one of its functions without it and the manufacturer built it. Many agent products, with an installed component and a required cloud backend, will fall on the CRA side of that line. Get a legal opinion on your own architecture.
What must the 24-hour early warning contain? For an exploited vulnerability: that it is being exploited, and where applicable the member states in which the product has been made available. For a severe incident: that it occurred, and whether unlawful or malicious acts are suspected. It goes through ENISA’s Single Reporting Platform to the CSIRT of the member state where you have your main establishment. The 72-hour notice and the final report add the detail.
Does the CRA require logging for AI agents? Annex I Part I point (l) requires products to provide security-related information by “recording and monitoring relevant internal activity, including the access to or modification of data, services or functions.” For an agent, that means an attributable record of which agent, on whose behalf, accessed which tool or data, and it needs to come from an enforcement point the agent cannot edit. Awareness of exploitation, which starts the 24-hour clock, depends on the same record.
What are the penalties under the CRA? Up to EUR 15 million or 2.5% of worldwide annual turnover for failing the essential cybersecurity requirements or the reporting duties, up to EUR 10 million or 2% for other obligations, and up to EUR 5 million or 1% for supplying incorrect or misleading information to authorities.

The regulator’s question

The people who wrote the CRA were thinking about routers, smart meters, and the software inside them. They were not thinking about a language model with a tool belt. But the law they wrote asks the questions that agent fleets are worst at answering: what is running, what can it reach, what did it do, and how fast can you say so.

LiteLLM’s bug was exploited in honeypots for weeks before it reached CISA’s list. Under the CRA, the clock would have started the day the honeypot lit up, and the first report would have been due the next day. Meeting that has less to do with how good the incident response team is than with whether the infrastructure already holds the answers.

So ask the question now, while it is cheap. If a component in your agent stack showed up on an exploited list tomorrow morning, which of your agents run it, what could each one reach, and what did they do last night? If the answer starts with “we would need to check,” the check is what to build.

Secure by design used to be a principle. In Europe, it now has a filing deadline.

The post EU Cyber Resilience Act: Europe Just Put Your AI Agents on a 24-Hour Clock appeared first on Tigera – Creator of Calico.

HIPAA Wasn’t Written for AI Agents. It Applies to Them Anyway

Alister Baroi — Fri, 02 Oct 2026 15:00:00 +0000

In short, healthcare is adopting AI agents faster than almost any other industry. More than 85% of Epic’s customers already use Epic AI, Epic’s Agent Factory will let every health system build agents of its own from 2027, and 43% of health systems were piloting agentic AI at the start of this year. Every one of those agents runs next to Protected Health Information (PHI), and PHI comes with rules that were not written for autonomous software but land on it anyway: minimum necessary access, audit controls, business associate agreements, a 60-day breach clock. This post maps those rules onto what agent infrastructure must provide (identity, per-request authorization, live inventory, an audit trail across every hop), then shows where Tigera Lynx fits and what it does not do. It is written for the platform and security leaders who will be asked to produce the record.

Healthcare was supposed to be the cautious one. Regulated to the bone, allergic to unvetted vendors, still running a fax machine somewhere in the basement. Instead, it is adopting AI agents faster than almost anyone.

At HIMSS in March 2026, Epic previewed Agent Factory, a visual builder for health systems to create, customize, and orchestrate their own AI agents inside the Electronic Health Record (EHR), and said that more than 85% of its customers are already actively using Epic AI. By its August user group meeting, four health systems were building agents in it without writing code. A month later, 25 health systems spent three days building clinical ones: an agent that reads NICU notes for early signs of necrotizing enterocolitis, another that checks medication refill requests against clinical protocol. Training for the first early adopters starts in October, and general availability is planned for 2027. Oracle’s Clinical AI Agent drafts notes and proposes next steps for physicians across more than 30 specialties, and in September it reached inpatient nurses. And underneath the vendor roadmaps sits a quieter layer: internal teams wiring LangGraph agents and MCP servers to FHIR APIs, because the pilot worked and the backlog is long.

The numbers behind that are worth holding in your head. Research from Microsoft and The Health Management Academy, published in early 2026, found that only 3% of health systems had agents in live workflows, while 43% were piloting or testing them. That is the shape of a wave a year before it breaks. The same study named governance as one of the three barriers health system leaders say they have not solved. Gravitee’s State of AI Agent Security survey, a vendor survey from February, found that 92.7% of healthcare respondents had seen a confirmed or suspected AI agent incident in the previous year, the highest share of any sector it asked.

None of this is recklessness, and it deserves better than a scolding. A scribe agent that gives a clinician two hours of their evening back is one of the clearest AI wins in any industry. The adoption is rational.

But every one of those agents runs next to PHI, which comes with rules that are older than the transformer, attach to the data rather than the technology reading it, and are about to get sharper, not softer.

Does HIPAA apply to AI agents?

Yes, fully, and without an AI chapter. HIPAA does not need one. The Privacy and Security Rules regulate covered entities and their business associates, and they attach to the data, so an agent reading a chart is governed exactly as a clinician reading a chart is. Four existing requirements land directly on agents:

Minimum necessary. 45 CFR 164.502(b) limits PHI access to what a specific task needs. A human scheduler looking up an address sees a demographics screen. An agent doing the same job through a service account that can read the full clinical record is a minimum necessary problem, whether or not it ever reads past the address field. Agents do not get scoped down by the login screen. They get exactly the credentials someone gave them.

Audit controls. 45 CFR 164.312(b) requires mechanisms that record activity in systems holding electronic PHI (ePHI): who accessed what, when, and on whose authority. EHR audit logs answer this for humans, because a human is logged in. An agent is not a user. It acts on behalf of a user, through credentials of its own, often through a chain: a patient asks an agent, the agent calls another agent, that one calls an MCP tool, the tool queries the record, and somewhere a language model sees the result. “Who” is now a chain, not a name, and the audit trail has to hold the whole chain.

Business associates. Any vendor that processes PHI on your behalf needs a Business Associate Agreement (BAA). Every LLM provider your agents can reach is on that list. If one team’s agent carries an API key for a provider you have no agreement with, its plumbing has quietly created a compliance exposure nobody signed off on.

Breach notification. When PHI goes somewhere it should not, the Breach Notification Rule starts a 60-day clock, and the first question is scope: what data, which patients, over what window. You cannot report what you cannot reconstruct. And the bill for getting this wrong is highest here of anywhere: IBM’s 2026 Cost of a Data Breach report puts the average healthcare breach at $6.64 million, the costliest industry for the thirteenth year running.

If the clock sounds theoretical, Australia just lived through the version without PHI in it. On 18 June, an OpenAI agent under internal evaluation found its way around the access blocks on a Services Australia portal that holds Medicare statistics and read non-public files. OpenAI discovered the intrusion on 11 August, told Services Australia on 10 September by emailing a public inbox, and the Prime Minister announced it on 24 September, calling both the delay and the manner “unacceptable”. No patient records were touched. Everything else about the story was the pattern: an agent that, in the Prime Minister’s words, did not accept no for an answer, and an operator that took two months to notice and another month to say so. OpenAI has since apologised, and after a further incident it paused training, evaluation, and tool-use inference on its most capable models. Put PHI in that story and the 60-day clock would have started in June, on the day the intrusion should reasonably have been known, not in August when someone noticed.

Figure 1: The OpenAI and Services Australia dates with HIPAA's 60-day window laid over them. Had it been PHI, the deadline would have passed 24 days before anyone was told.

One agent makes each of these a manageable exercise in process. Ten agents make it a spreadsheet nobody enjoys maintaining. A hundred agents, some built in a vendor’s visual builder, some homegrown, some a resident’s experiment that never got decommissioned, make it impossible to do by hand. That is not a prediction about a distant future. Agent Factory’s entire pitch is that your departments will build their own.

How is the HIPAA Security Rule changing for AI?

The rule is not changing for AI specifically, but it is tightening in exactly the places agents stress. In January 2025, HHS proposed the first major rewrite of the Security Rule in over a decade. The final rule has slipped: OMB’s regulatory agenda now targets July 2027, nearly 5,000 comments are still under review, and hospital groups, including the American Hospital Association, have asked for the proposal to be withdrawn. Proposals change before they land. The direction will not. Three items in the proposal read as if they were written with agent fleets in mind:

A technology asset inventory and a network map showing how ePHI moves through your systems, reviewed at least annually. Every agent, every MCP server, every LLM endpoint is an asset on that map. A map drawn by interviewing teams is stale before the ink dries. Agents get built in an afternoon.
Mandatory, auditable controls in place of “addressable” ones: encryption, multi-factor authentication, segmentation. The era of arguing that a control was not reasonable for your environment is closing.
Annual written certification from business associates that safeguards are actually deployed. Your vendors will be certifying to you, and your auditors will expect you to know which vendors sit in the data path at all.

Nobody is waiting for the rule, either. The Office for Civil Rights (OCR) is enforcing the current rule’s risk analysis requirement hard, through a Risk Analysis Initiative that passed its fourteenth settlement in June, and a risk analysis is only as good as the inventory it starts from. The Ambry Genetics settlement in September is worth reading with agents in mind: besides the missing risk analysis, OCR faulted the company for never assigning unique IDs to track who touched ePHI. That was a finding against a workforce of humans. A fleet of agents sharing one service account fails the same test, for the same reason.

HHS ran a request for information on accelerating AI in clinical care over the winter and reported in June that what the sector asked for most was practical guidance on governing AI and tools to evaluate what vendors sell. It says the answers will shape how it regulates, reimburses, and funds AI in the clinic. States are moving faster. California’s legislature sent the governor AB 1979 in September, which would require any company whose AI health application reaches medical records, consumer chatbots included, to comply with the state’s medical confidentiality law, and the governor had until 30 September to act on it.

Whichever year the final rule takes effect, the gap between “we have a policy document” and “here is the map, and here is the log” is exactly where findings live.

What does HIPAA compliance require of AI agent infrastructure?

Put the current rule and the proposed one side by side, then translate each requirement into what it means once agents are in the data path:

The rule asks	With agents, that means	So the infrastructure must provide
Minimum necessary access (164.502(b))	An agent touches only the PHI its current task needs, not everything its credentials allow	Per-request authorization with default deny, not standing access
Audit controls (164.312(b))	Every access attributed to a specific agent, acting for a specific person, through a specific tool and model	An identity-attributed audit trail across every hop, not just the EHR login
Asset inventory and ePHI network map (proposed)	Every agent, MCP server, and LLM provider is on the map, including the ones nobody registered	Continuous discovery of agents, not an annual questionnaire
BAA coverage for vendors in the data path	Agents can reach only the LLM providers you actually have agreements with	Central control over provider access, with credentials held outside the agent
Breach notification within 60 days	Scope a suspected leak: which agent, what data, which direction, what window	A reconstructable record of agent traffic with policy decisions attached

Figure 2: The same table drawn as one request: five hops, a checkpoint at each, and one agent nobody registered.

Read column three again. Nothing in it is paperwork. Identity, authorization, discovery, audit: every row is infrastructure, and none of it can be delegated to the agent itself, because controls the agent can override are not controls. We have made that argument all year, from the accountability crisis through why network policies, API gateways, and RBAC cannot close the gap. Healthcare is where it stops being an argument and becomes an audit finding.

Notice, too, what the table does not contain. It does not ask the model to behave. Prompt injection is unsolved, and an agent that has been talked into exporting a patient list will do so with perfectly valid credentials. The five pillars of AI agent accountability we published earlier this year (traceability, authorization provenance, identity and ownership, policy at scale, human oversight) were written for any regulated industry. Lay them over that table and each pillar answers one or two rows. HIPAA just happens to have written the pillars into law before anyone called them that.

Where does Tigera Lynx fit?

This is the gap Tigera Lynx was built to close, and it maps onto that table almost row for row.

Every agent, MCP server, and LLM provider registers with Lynx and carries a verifiable identity (SPIFFE, OIDC, or a pod-owner binding for agents you cannot modify), including who the agent is acting for, so “which agent, on whose behalf” is a fact attached to every request rather than a forensic project after the incident. Authorization runs through Cedar policy at the gateway: default deny, evaluated on every request, with the caller’s identity, the target, and the operation as context. For MCP traffic the policy can see the operation, the tool name, and the tool arguments, so “this agent may call lookup_appointment but never export_patient_list” is one policy statement, and a call a policy marks as requiring human approval waits until a person signs off. That is minimum necessary expressed as executable policy instead of a training slide.

LLM provider credentials are injected at the gateway and never live inside the agent, which means the set of providers your agents can reach is exactly the set you have BAAs with, and a prompt-injected agent has no key to leak. Each token the gateway mints is addressed to one destination, so a token stolen from one hop is worthless at the next.

Agents that never registered, the experiment nobody decommissioned or the sidecar nobody audited, are detected from the node itself. An eBPF detector watches for connections to LLM endpoints, including a watchlist of your own self-hosted model endpoints, and inspects running processes for agent frameworks, then classifies each workload as sanctioned, shadow, or unknown with the evidence shown beside the verdict. That is the difference between an asset inventory you assert and one you can defend. A workload that should not be there can be quarantined on the spot, at the gateway and in the kernel, so it can no longer write to disk, launch processes, or open an outbound connection. The same detector now ships as a package for plain Linux hosts, because not every agent in a health system runs on Kubernetes.

And every request lands in the Agent Trail with the caller, the human authority, the tool, the policy decision, the token exchanged at each hop, and the model that actually served the call: your 164.312(b) mechanism and your breach-scoping record in one place, written by the enforcement plane rather than narrated by the agent.

To be precise about the claim: no software makes you HIPAA compliant, Lynx included. Compliance is contracts, risk analysis, process, and training, and the BAA with your LLM provider is a document your legal team signs, not a policy the gateway enforces. What Lynx provides is the layer those processes quietly assume exists: the ability to say, with evidence, which agents exist, what each one can touch, and what each one did.

Frequently asked questions

Are AI agents subject to HIPAA? Yes. HIPAA attaches to PHI and to the covered entities and business associates that handle it, not to the technology in between. An agent that reads, creates, or transmits PHI is bound by the same minimum necessary, audit control, and breach notification requirements as any workforce member or system.
Is my LLM provider a HIPAA business associate? If PHI reaches it, yes, and you need a BAA before it does. Most major providers offer one on specific enterprise tiers, and de-identified data falls outside the rule. Signing the BAA is the easy part. The engineering problem is guaranteeing that no agent can reach a provider you have not signed one with, and that is a routing and credential control rather than paperwork.
Can an AI agent be “HIPAA compliant”? Not on its own. Compliance is a property of the organization and its processes. What you can make an agent is governable: it has a verified identity, its access is scoped per request, its traffic is logged with attribution, and it can be stopped. That is what an auditor can check.
What does an AI agent audit trail need to contain? For every access to ePHI: which agent, acting on whose behalf, through which tool or API, against which policy decision, at what time, and which model saw the result. A log the agent writes about itself does not count. The record has to come from an enforcement point the agent cannot edit.
What is the minimum necessary standard for AI agents? An agent should be able to reach only the PHI its current task requires, evaluated at the time of the request. Standing access through a broad service account violates the spirit of the standard even when the agent behaves, because the exposure exists whether or not it is exercised.
Does the proposed HIPAA Security Rule apply to AI agents? It does not mention them, but its asset inventory, network map, mandatory encryption and MFA, and business associate verification requirements apply to every system that touches ePHI. Agents, MCP servers, and LLM endpoints are systems. The final rule is now expected in 2027 and may change, but the current rule’s risk analysis requirement, which OCR is actively enforcing, already needs a complete inventory to satisfy.

The auditor’s question

The vendors will keep shipping agents, and clinicians will keep adopting anything that gives them their evenings back. The industry conversation has already moved from “should we deploy agents” to “how do we validate and govern them.” That is the right conversation, and it comes with a deadline attached: the day an auditor, or a breach, asks you to produce the record. OpenAI got that question in August. Its answer reached Services Australia a month later, by email, in a public inbox.

So ask the question now, while it is cheap. When someone asks which agents could read patient records last quarter, which ones actually did, on whose authority, and which models saw the results, what will you hand them?

In most industries, an unaccountable agent is a risk to manage. In healthcare, it is a finding waiting for an auditor.

_ Lynx is Tigera’s security and governance platform for AI agents on Kubernetes: identity, policy, detection, and audit for every agent in your cluster. Read How Lynx Works or request early access at tigera.io/demo/?product=lynx._

The post HIPAA Wasn’t Written for AI Agents. It Applies to Them Anyway appeared first on Tigera – Creator of Calico.

Managing Claude Code Sessions Through Lynx

Alister Baroi — Wed, 30 Sep 2026 15:00:00 +0000

At Tigera, we spend a lot of time thinking about agent security: identity, policy, runtime controls, and the record left behind after an agent acts.

Coding agents create an interesting problem because, in most organizations, they didn’t arrive through the front door.

Few companies ran a platform evaluation and rolled Claude Code out to 500 developers. Developers installed it themselves. By the time security and platform teams started asking how coding agents should be governed, they were already running on laptops with access to source code, credentials, SSH keys, kubeconfigs, internal services, and whatever else the developer could reach.

The long-term answer is increasingly clear I think: move coding agents into isolated environments you control.

Anthropic’s sandboxing work draws filesystem and network boundaries using OS primitives such as bubblewrap and seatbelt. Its reference devcontainer includes an egress firewall. Docker has introduced sandboxes for running coding agents, and Kubernetes-based approaches can add stronger workload isolation, network policy, and disposable development environments.

That direction makes sense.

Isolation governs what an agent can do. A gateway governs what it can send.

And unlike a complete move to remote development environments, the second boundary is something you can introduce today.

Start with one environment variable

Claude Code allows its model endpoint to be configured through the environment.

Instead of connecting directly to Anthropic, point it at the Lynx gateway.

export ANTHROPIC_BASE_URL="https://<lynx-gateway>/llm/anthropic"

That’s the entire change on the developer’s machine.

No endpoint agent. No daemon. No certificate installation.
No rebuilt development environment.

The next model turn goes through Lynx, and every model turn after it does too.

That small change turns otherwise disconnected model calls into something more useful: a coding session that can be observed, governed, and recorded.

Coding sessions list in Lynx

Coding session summary in Lynx

What this gives you

Instead of seeing another HTTPS connection from a developer laptop to an AI provider, the platform can start answering important questions:

Which coding sessions are using AI?
Which model providers and models are they using?
What information is being sent to those models?
Which policies were applied?
Which MCP tools were called through the gateway?
Did the session spawn sub-agents?
What happened during a particular coding session?

That is a significant improvement over an unmanaged agent talking directly to a model provider.

What this does not do

A gateway gives you a strong answer to what left the building, not what happened on the machine.

Local tools run on the laptop and do not transit the gateway.
Example: Claude Code reads a file, edits source code, or runs a shell command locally.
Some of that activity may appear in model traffic (e.g., tool calls, results, retrieved files, or other context).
That is observation, not enforcement.

A governed laptop is not a sandbox

Routing Claude Code through a gateway does not suddenly give Lynx control over the developer’s machine.

Every model turn that transits the gateway can be governed.
MCP calls can also be governed when the MCP server is reached through the gateway.
Local tools are different. The gateway isn’t on that execution path and cannot prevent it.

But that is observation, not enforcement.

What a governed coding session looks like

From unmanaged laptop to governed sandbox

Learn more about Lynx →

The post Managing Claude Code Sessions Through Lynx appeared first on Tigera – Creator of Calico.

The Clearinghouse For AI Agents Has A Blind Spot

Dillon Barry — Wed, 23 Sep 2026 09:11:12 +0000

Jamin Ball’s recent piece, “Systems of Record Won the SaaS Era — Clearinghouses Will Win the Agents Era,” is the cleanest articulation I’ve seen of where the durable moat goes next. His argument is simple and, I think, correct: the SaaS era rewarded whoever owned the system of record, and the agent era will reward whoever owns the clearinghouse. This is a new seat that sits between mutually-distrusting agents and decides which one is cleared to act, on what data, with what limits, and can prove what happened after the fact. The lock-in moves from your data to your permissions. The source-of-truth era becomes the source-of-permission era.

I agree with all of it. Governance has graduated from a compliance checkbox at the end of the sales cycle to the first question a CIO asks. Once agents act autonomously and eventually spend real money, “is the model good?” stops being interesting now that every model is good enough and “can I see what every agent did, set policy on what it can touch, and prove it to my auditors?” becomes the whole conversation. That seat is strategic real estate. Whoever holds it is hard to dislodge.

But there’s an assumption buried in the clearinghouse metaphor that’s worth dragging into the light, because the entire model rests on it.

A clearinghouse only clears what comes to it

In financial markets the metaphor works because participation is mandatory. You cannot settle a trade on a regulated exchange without going through the clearinghouse. The seat is powerful precisely because there is no path around it. Every transaction is, by construction, a transaction that presents itself for clearing.

Agents are not like that. Nothing about the architecture forces an agent to route its actions through a governance seat. An agent calls a model endpoint directly. It opens a connection to a tool server that someone stood up last week. It talks to another agent over a side channel or a message board as in the Hugging Face attack. A developer spins up an autonomous workflow in a corner of the infrastructure that the platform team has never inventoried. None of that traffic announces itself at the toll booth. And the receipt the clearinghouse keeps is, by definition, a record only of the transactions that came to the desk.

This is the gap. The clearinghouse is an on-path construct, it governs the actions that pass through it. But in a real enterprise, the actions you most need to govern are frequently the ones that don’t. The shadow agent nobody registered. The model call that skipped the gateway. The agent-to-agent handoff that never generated a clearing event. A governance model that only sees submitted transactions is blind to exactly the population that represents the risk.

Every example in Jamin’s framing, the data platforms winning “from below,” the productivity suites winning “from above” assumes the agent shows up to be cleared. The most dangerous agents are the ones that never do.

Source of permission needs a source of truth

So the clearinghouse seat, as described, is necessary but not sufficient. To actually hold it, you need something underneath it that the metaphor skips over: an independent source of truth about what agents are doing, whether or not they chose to clear through you.

Four capabilities make that real, and none of them are “clearing” in Jamin’s sense.

The first is off-path visibility. You need to observe agent traffic at a layer the agent can’t opt out of, that is the network and runtime substrate the workload actually executes on, so that the calls that bypass the gateway are still seen. A receipt tells you what cleared. Visibility tells you what happened. Those are different data sets, and the difference between them is your risk surface.

The second is discovery. The clearinghouse clears the agents that present themselves; it does not go find the ones that didn’t. You need to enumerate the agents running in your environment without relying on them to register, because the unregistered ones are the point.

The third is behavioral sensing. Clearing is transactional: permit or deny, one action at a time. But a single permitted action can be perfectly legitimate while a sequence of them is an exfiltration pattern. You need continuous analysis of what an agent does over time, not just a verdict at each gate.

The fourth is identity issuance, not just verification. A clearinghouse authorizes a credential at the door. But in a multi-hop agent world where agent calls agent calls tool calls model, you need every hop to carry attributable, minted identity, so the audit trail Ball rightly prizes is actually trustworthy end to end rather than a chain of “trust me” assertions.

Where I’d draw the line

I want to be careful not to overclaim the inverse, either. The clearinghouse thesis gets two things exactly right that the visibility layer does not replace. Owning memory , what agents know and context , what they see and how it’s served, is a genuinely strategic seat, and it belongs to the data players. A visibility-and-governance layer governs access to those things; it doesn’t store them, and it shouldn’t pretend to. The division of the map is this: the data platforms own memory and context, and the governance layer owns permission, enforcement, and the source of truth about behavior.

And the multi-vendor angle Jamin references, “the clearinghouse across clearinghouses,” the neutral seat that no incumbent can credibly occupy because Microsoft will never neutrally govern Salesforce’s agents — is the most valuable position of all, but it’s also the least mature. Neutrality is easy to assert and hard to ship, especially across the messy reality of where agents actually run today. Anyone claiming it now is selling a roadmap.

The reframe

So here’s where I land. Jamin is right that the moat moves from the system of record to the clearinghouse, and from your data to your permissions. But a permission you can’t enforce on traffic you can’t see is a permission in name only. The source-of-permission era doesn’t replace the source of truth, rather it depends on one. The seat goes to whoever can both decide what an agent is allowed to do and independently observe what it actually did, including all the times it never asked.

The clearinghouse is the right metaphor. It just needs eyes outside the building.

Written by Alain Mayer

The post The Clearinghouse For AI Agents Has A Blind Spot appeared first on Tigera – Creator of Calico.

HITL for autonomous agents: Where does the human go?

Alister Baroi — Thu, 17 Sep 2026 00:49:44 +0000

Human approval is easy when you are sitting in front of the agent. For an agent running by itself in a cluster, almost none of that holds.

You’re in a meeting and your agent is running in a cluster. It has a service account, it has been asked to keep a service healthy, and it has just worked out that the right fix is to roll back a database migration. Nobody is watching it. That was rather the point of deploying it. You want to get notified to approve such an important action.

This is a different problem from the one most people picture when they hear human in the loop. If you use a coding assistant, you already have human oversight and it costs almost nothing: the agent shows you a diff, you read it, you approve. That works because you are already there — at a keyboard, with the context in front of you, in the same second the agent needs an answer.

An autonomous agent has none of that. There is no session to interrupt. The person who should decide is asleep, or in a meeting, or on a plane. The approval has to travel out of the cluster, reach a phone, and come back. And the request has to survive however long that takes.

Almost everything that is hard about human oversight for autonomous agents follows from those two sentences. So the useful question is not whether to have a human – MCP’s tools specification recommends keeping a person able to deny a tool call, and every agent security note I have read says some version of the same thing. The useful question is where that human sits, and what happens to the request while it waits.

Four places to put the human

Four different layers of the stack have each grown an answer, and they are not the same answer.

At the identity layer , the agent asks an identity provider for permission. The provider reaches you out of band, and your approval mints a short-lived token scoped to that one action. This is OpenID Connect’s CIBA, and it is the most mature of the four — Auth0, Okta and others have productized it, and Christian Posta has made the case for it as the natural home for agent oversight.

At the protocol layer , the tool server pauses and asks a question back through the client. MCP calls this elicitation and specifies it properly. It also assumes there is a client with a person attached to it, which puts us back at two in the morning.

At the infrastructure layer , a gateway in the request path holds the call, notifies someone, and then releases or refuses it. Nothing in the agent changes, because the agent never learns it was held.

At the runtime layer , the agent framework pauses itself and checkpoints. LangGraph’s interrupt() and Temporal’s durable execution both work this way: the pause is saved state rather than a blocked thread, so it survives a crash or a deploy.

These are not four implementations of one feature. CIBA answers what authority does this action now carry. Elicitation answers what does the user want to happen. A gateway hold answers should this request proceed. A durable interrupt answers how does the work survive the wait. We started out treating them as alternatives and had to stop.

The wait is the hard part

A person takes minutes. A request does not.

Hold the call open at a gateway and you get approval with no change to the agent at all — which is the whole reason to do it there — but you are now racing every timeout between the caller and you. Client timeouts, proxy idle timeouts, load balancer limits. You can raise them, but you raise them for all traffic, not just the held kind. Two minutes is generous by network standards and insulting by human ones.

Return immediately instead, mark the call pending, and let the caller poll. TrueFoundry describe this shape well and are honest about the cost: the client has to understand the convention and decide when to retry. That is a change to every agent you were trying to leave alone.

Or push the pause into the runtime, where LangGraph and Temporal have already solved it properly. The wait is checkpointed, it survives a restart, and holding for an hour costs nothing. It also requires that you own the agent’s runtime, which rules out the third-party agent and the one you did not write.

Each design pays one of those three prices. We pay the first.

What does an approval prove?

When the log says the action was approved, what has actually been established?

In the strong version: this named person, authenticated at this strength, saw this specific request with these specific arguments, and decided at this time. In the weak version: a push went somewhere and something came back green.

In our case it was the notification channel that opened the gap between the two. We integrated Cisco Duo as an approval backend, because it is already deployed at the kind of enterprise we work with and the push-to-phone experience is exactly right for the two-in-the-morning case. But Duo’s status response tells us the transaction was approved. It does not tell us who approved it, or on what device. So the approver we can record is the one we were configured to ask, not the one who actually tapped. Our first-party path, where someone answers through an authenticated session, can name the individual. The Duo path cannot.

That is not a complaint about Duo, which is doing exactly what a second-factor product is designed to do. It is a warning about reading approval logs — and it is the strongest argument for the CIBA model, where the approver’s identity travels inside the token rather than sitting in a log line beside it.

If you are choosing a design, ask what it can prove, not what it can block. Blocking is the easy half.

Approval should not create permission

There is a design fork here, and we came down firmly on one side of it.

When the human says yes, does that create permission, or does it satisfy a condition in permission that already existed? CIBA takes the first path by construction: the approval mints a token, and the token is new authority. That is clean and portable, and it also means a leaked token is authority in someone else’s hands until its window closes.

We took the second path. In Lynx a gate is an ordinary Cedar forbid rule with an annotation and a condition attached:

@approvalRequired("Approve transfer_funds on treasury")
forbid (principal, action, resource)
unless { context.approval.granted };

The annotation marks the rule as worth holding and carries the prompt the person reads. The unless clause is what an approval satisfies. Both halves are required, and our linter rejects one without the other.

The consequence is the part I care about: the gateway never turns a deny into an allow. If the agent was not permitted to reach that server in the first place, no amount of approval gets it there, and the request is refused immediately rather than paging someone at 2am to rubber-stamp something that was never going to work. Approval is one input to a decision still made by policy. It is not a manual override, and I would not be comfortable shipping one that was.

The person holding the phone

Once a human is in the path, that human is a target, and the attacks are all about what the notification says. Most of what appears on an approval screen is influenced by someone else: the tool name comes from a server that may be compromised, the arguments come from a model that may have been steered by a retrieved document, and MCP annotations are explicitly hints rather than facts. Render all of that as one undifferentiated block and you are asking a half-awake person to spot a lie on a small screen. So we treat the rendering as a security control — authenticated caller identity first, written from what the gateway verified; caller-supplied arguments namespaced under an arg. prefix so nothing can pose as provenance; text flattened and truncated. None of it is clever. The alternative is a phishing surface with a corporate logo on it.

The quieter failure is fatigue. If the answer is always yes, the gate has stopped being judgment and become latency with an audit trail. Every change that reduces the number of prompts — batching, scoping, remembering a decision for a window — is also a change that puts fewer requests in front of a person. I do not think there is a way out of that tension, only a way to be deliberate about where you sit in it.

Where we have got to

We build on agentgateway as our data plane, and Solo’s enterprise build already ships human approval with notifications out to Slack, email and mobile. This is not new ground, so I would rather say what we learned than claim novelty.

We hold at the gateway, gated by Cedar, so that an agent nobody wants to modify can still be governed. We drive Duo’s asynchronous push for the out-of-band decision. That is deliberately not CIBA — Duo has no backchannel authorization endpoint and no CIBA grant — but the shape is the same: client-initiated, decoupled device, poll for a decision. We built the seam to that shape so a real CIBA backend can sit behind it later without the policy surface changing. Since MCP’s specification declines to standardize an approval workflow, borrowing the shape of the standard next door seemed better than inventing a fifth one.

What we have not solved: the hold is synchronous, so it carries the timeout cost above. There is a single global approver, where the problem wants routing by resource and by policy. It covers MCP tool calls and not yet the other paths an agent can take. And the trail does not yet surface the way it needs to for someone reconstructing an incident months later.

I am writing this before those are fixed rather than after, because the feature is not the interesting part. The interesting part is that an identity specification, a protocol specification, a set of gateways and a set of agent frameworks all produce something different when a human says yes, and none of them drops into the place of another. That is worth working out now, while the designs are still soft enough to change.

Agents will keep asking for permission to do consequential things while nobody is looking. We should be more precise about what we mean when we say a human approved it.

Learn more about Lynx →

Further reading: the OpenID CIBA specification; Christian Posta on agent autonomy and CIBA; Auth0’s asynchronous authorization for agents; TrueFoundry on tool approvals at the gateway boundary; LangGraph durable execution and Temporal’s LangGraph plugin.

The post HITL for autonomous agents: Where does the human go? appeared first on Tigera – Creator of Calico.

AI Agents on Kubernetes 101: From Laptop Script to Production Pod

Alister Baroi — Tue, 08 Sep 2026 20:16:27 +0000

In short, this is a beginner’s guide to deploying an AI agent on Kubernetes. You will containerize an agent, store its API key as a Kubernetes secret, write a deployment with health probes and resource limits, expose it with a service, and lock down its network egress, in that order, with a working manifest at every step. On a local kind cluster the whole walkthrough takes about an hour. At the end: the six mistakes almost every first agent deployment makes, and the questions a 101 deployment leaves open.

Every AI agent starts life the same way; a Python script on someone’s laptop, an API key in a .env file, a while loop around an LLM call. It works, it demos well. Then someone with a budget says “ship it,” and you, the engineer closest to the script, get to figure out what shipping an agent actually means.

This guide is that path, walked slowly. It assumes you know what a container is and have met kubectl at least once, and it assumes nothing about agents. By the end you will have an agent running in a cluster with its key in a Secret, its resource usage capped, its health checked, and its network access reduced to the short list of places it has any business calling. That is still short of production-grade governance, but it is a deployment you could defend in a code review, which is more than most agents get.

Why is Kubernetes a good place to run AI agents?

The short answer is, because an AI agent is a workload that needs supervision, and Kubernetes is the most widely used system for supervising workloads. It restarts the agent when it crashes, caps how much CPU and memory it can consume, injects its credentials (e.g. API keys) at runtime instead of baking them into the code, and describes all of it in version-controlled YAML manifest your team can review. Just as important, Kubernetes gives you control over the agent’s environment, such as, what it can reach on the network, what identity it carries, what happens when it misbehaves. That matters more for agents than for ordinary services, because an agent’s behavior is decided at runtime by a model, so the rules have to live outside the agent, where no prompt can talk its way past them. That argument deserves more than a paragraph, so we gave it a full post: The Safest Place to Run an AI Agent Is a Cluster That Doesn’t Trust It makes the case with a year of real incidents as evidence.

How is deploying an agent different from deploying a web service?

Mechanically, it barely is. An agent is a long-running process that speaks HTTP; Kubernetes has been running those for a decade. The difference is in what the workload does with its freedom:

A web service follows its code. An agent follows a model. You can read a service’s code and know its behavior. An agent’s next action depends on whatever lands in its context window.
An agent holds credentials and acts. It calls APIs, queries databases, sends messages. A wrong answer is a bug; a wrong action is an incident.
Every input is potentially an instruction. Prompt injection is unsolved. Text the agent reads, from a user, a document, or another agent, can try to redirect it.

So the deployment steps below are the same ones you would use for any service. The difference is emphasis: the steps most tutorials treat as optional hardening (secrets, limits, egress control) are, for agents, the point.

What you need before you start

Four things, all free:

A cluster. Locally, kind runs Kubernetes inside Docker: kind create cluster --name agents. Any managed cluster (EKS, GKE, AKS) works the same way.
kubectl and Docker , installed and talking to that cluster.
An agent that speaks HTTP. Any framework is fine. LangGraph, CrewAI, Google ADK, etc all wrap into a web server; we compared them in Six AI Agent SDKs for Enterprise Kubernetes. The walkthrough uses a generic Python agent served by FastAPI on port 8080, with two routes: POST /chat for work and GET /healthz that returns 200 when the process is up. If your agent lacks a health route, add one first. It is five lines, and step 4 depends on it.
An LLM API key. From whichever provider your agent calls.

One namespace keeps the experiment contained:

kubectl create namespace agents

Step 1: Containerize the agent

Kubernetes runs containers, so the script becomes an image. A minimal Dockerfile for a Python agent:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
USER 1000
EXPOSE 8080
# change the command (to run agent) according to its framework documentation
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

Two lines here are security decisions. USER 1000 runs the agent as a non-root user, so a compromised agent is not root inside its container. And notice what is absent: no API key. The key never enters the image. An image is copied to registries, cached on nodes, and pulled by anyone with access; a key baked into an image is a key you have already leaked, you just don’t know to whom yet.

Build it and, for kind, load it into the cluster:

docker build -t support-agent:0.1.0 .
kind load docker-image support-agent:0.1.0 --name agents

On a real cluster you would push to a registry instead. Either way, tag with a version (e.g., 0.1.0), never latest. You want to be able to say exactly what is running, and roll back to exactly what was running before.

Step 2: Put the API key in a Secret

The.env file’s job is taken over by a Kubernetes Secret:

kubectl -n agents create secret generic support-agent-secrets 
  --from-literal=ANTHROPIC_API_KEY='sk-ant-...'

The Secret lives in the cluster, is delivered to the pod as an environment variable at start time, and can be rotated without rebuilding the image. Two honest caveats for later: Secrets are base64-encoded, not encrypted, so real clusters enable encryption at rest and restrict who can read them with RBAC; and the agent’s own process can still read this variable, which matters once you worry about prompt injection. Hold that thought for the end of the article.

Step 3: Write the Deployment

The Deployment is the contract: which image, how many copies, what resources, what health checks. Save this as deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: support-agent
  namespace: agents
  labels:
    app: support-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: support-agent
  template:
    metadata:
      labels:
        app: support-agent
    spec:
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
      containers:
        - name: agent
          image: support-agent:0.1.0
          ports:
            - containerPort: 8080
          envFrom:
            - secretRef:
                name: support-agent-secrets
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              cpu: "1"
              memory: 512Mi
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 15
            periodSeconds: 20
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]

Apply it with kubectl apply -f deployment.yaml, then read it back top to bottom, because every block answers a question a reviewer will ask:

automountServiceAccountToken: false: By default every pod gets a token for the Kubernetes API mounted into its filesystem. Your agent doesn’t need to talk to the Kubernetes API, so it doesn’t get the token. For an agent this is not a nicety. It is the difference between “prompt injection stole a chat log” and “prompt injection got a foothold in my cluster’s control plane.”
resources : Requests are what the scheduler reserves; limits are the ceiling. Agents fail in loops. A model that decides to retry forever, or to summarize a document by reading it into memory in one piece, will eat a node if you let it. Limits turn “the cluster is down” into “one pod got throttled.”
readinessProbe and livenessProbe : Readiness controls whether traffic is sent to the pod; liveness restarts it when the process wedges. Without them, Kubernetes considers a hung agent healthy forever.
envFrom.secretRef : The key from step 2 arrives as an environment variable. Code reads ANTHROPIC_API_KEY exactly as it did from the .env file. Nothing about the agent’s code had to change.

Check it comes up:

kubectl -n agents get pods
kubectl -n agents logs deploy/support-agent

Step 4: Expose it with a Service

Pods are ephemeral and their IPs change. A Service gives the agent a stable name:

apiVersion: v1
kind: Service
metadata:
  name: support-agent
  namespace: agents
spec:
  selector:
    app: support-agent
  ports:
    - port: 80
      targetPort: 8080

Now anything inside the cluster can reach the agent at http://support-agent.agents.svc. From your laptop, test through a port-forward:

kubectl -n agents port-forward svc/support-agent 8080:80
curl -X POST localhost:8080/chat -H 'Content-Type: application/json' 
  -d '{"message": "hello"}'

If you get an answer, you have an AI agent running on Kubernetes. Most tutorials stop here. Do not stop here.

Step 5: Close the doors it doesn’t need

Right now your agent can open a connection to anything: every pod in the cluster, every address on the internet. For a web service that is untidy. For an agent it is the whole attack surface, because the standard end of a prompt-injection chain is exfiltration: the agent is talked into sending data somewhere it should never call. A NetworkPolicy makes egress deny by default:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: support-agent-egress
  namespace: agents
spec:
  podSelector:
    matchLabels:
      app: support-agent
  policyTypes: ["Egress"]
  egress:
    - to:
        - namespaceSelector: {}
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    - ports:
        - protocol: TCP
          port: 443

This allows DNS lookups plus outbound HTTPS, and nothing else; a connection to another pod, or plaintext HTTP to anywhere, now fails at the network layer (Note for kind users: enforcing NetworkPolicy requires a network plugin that implements it, such as Calico; the kind default does not).

The policy still has one hole, and it is an instructive one: “TCP 443 to anywhere” lets the agent reach any HTTPS endpoint on the internet, including an attacker’s. Vanilla NetworkPolicy speaks IPs and ports; it cannot say “only api.anthropic.com.” Narrowing egress to named destinations takes either a DNS-aware policy engine (Calico can do this) or, better for agents, an egress gateway that all agent traffic must pass through. Remember this gap. It is where 101 ends and the last section of this article begins.

The six mistakes every first agent deployment makes

The walkthrough above quietly avoided all six. Here they are in the open, because you will meet them in other people’s manifests:

Mistake	Why it hurts an agent especially	Instead
API key baked into the image	Images get pulled, cached, and shared; the agent’s key is its power	Kubernetes Secret, injected at runtime
No resource limits	Agents fail in loops; one runaway loop starves the node	Set `requests` and `limits`
`latest` image tag	You cannot say which agent behavior is running, or roll back	Version tags, ideally digests
No health probes	A wedged agent looks healthy and keeps receiving work	`readinessProbe + livenessProbe`
Default ServiceAccount token mounted	Injected agent inherits a path to the Kubernetes API	`automountServiceAccountToken: false`
Unrestricted egress	Prompt injection ends in exfiltration over open egress	Default-deny NetworkPolicy, then allowlist

Read the middle column again. None of these is an exotic agent problem; they are ordinary Kubernetes hygiene. Agents just raise the price of skipping them.

What a 101 deployment still can’t answer

You now have one agent, contained and supervised. Before you call it production, try to answer three questions about it:

Who is this agent? Your cluster knows it as a pod with a label. Nothing cryptographically distinguishes it from any other workload, so nothing downstream can grant it permissions as an agent, or refuse an impostor.
What exactly is it allowed to do? Your NetworkPolicy says “HTTPS, somewhere.” It cannot see that the agent is calling an MCP tool, or with what arguments. “May this agent call delete_records on the billing server” is not a question any layer you have deployed can even parse. We walked through why the stock building blocks stop short in the accountability gap post.
What did it do last Tuesday? You have pod logs, written by the agent itself. An agent’s own narration is not an audit trail; the best-documented case of an agent deleting a production database came with the agent confidently reporting that rollback was impossible. It was not.

And this is with one agent. The moment there are two, they start talking to each other (that is theA2A protocol), and the questions multiply by every pair.

That missing layer (identity, per-request authorization, and audit across a fleet of agents) is what Tigera Lynx adds on top of exactly the deployment you just built. Lynx gives every agent a SPIFFE or OIDC workload identity, routes agent-to-agent, MCP, and LLM traffic through a gateway that authorizes each request under Cedar policy, and records every hop in an Agent Trail the agent cannot edit. Recent releases sharpened the credential story from this post’s step 2: provider keys attach at the gateway instead of living in the pod’s environment, per-hop tokens are minted with an audience of one target so a stolen token is nearly worthless, and an OAuth authorization-code flow can issue per-user credentials. Policies can require human approval before a risky MCP call proceeds. On the node, an eBPF detector spots agents nobody registered and can quarantine a compromised one down to blocking its network writes at the kernel, and a policy playground in the UI lets you test a Cedar policy against simulated requests before it ever gates real traffic.

Frequently asked questions

Do I need a GPU to deploy an AI agent on Kubernetes? No. An agent is orchestration code that calls a model over an API; the GPUs live with the model provider. You only need GPU nodes if you self-host the model itself, which is a separate project.
Is Kubernetes overkill for a single agent? For a weekend experiment, yes; run it locally. But the reasons to move to Kubernetes (restarts, secrets, resource caps, network control) show up the first time the agent touches real credentials or real users, which happens earlier than most teams expect.
Can I do all of this on my laptop? Yes. Everything above runs on a kind cluster in Docker, including the NetworkPolicy if you install a plugin like Calico that enforces it.
Which agent framework works best on Kubernetes? Any framework that can serve HTTP deploys the same way. The differences show up in observability, state handling, and protocol support; our six-SDK comparison covers them.
How do I scale an agent to more replicas? Set replicas: 3 and Kubernetes load-balances across them, but only if the agent keeps its conversation state outside the pod (a database or cache), since any replica may serve the next request. Stateless agents scale for free; stateful ones need that refactor first.
How is deploying an agent different from deploying a model? A model deployment serves inference (weights on GPUs behind an endpoint). An agent deployment runs the loop that calls models and tools to pursue a goal. This guide covers the agent; most teams consume the model as a managed API.

Ship the pod, then ask the harder question

Back to that laptop script. The distance from .env file to the deployment in this article is five manifests and an afternoon, and every step was ordinary Kubernetes, applied with an agent’s failure modes in mind. That is the good news: you do not need new infrastructure to give an agent a safer home than a laptop. You need the infrastructure you already have, used deliberately.

The harder question arrives with agent number two: when they start acting on each other’s behalf, who is checking identity at the door, and where is the record? Deploying the agent was the easy 101. Trusting it is a course Kubernetes alone does not teach.

Learn more about Lynx →

The post AI Agents on Kubernetes 101: From Laptop Script to Production Pod appeared first on Tigera – Creator of Calico.

Meet Mylo: An AI-native way to work with Calico

Alister Baroi — Thu, 03 Sep 2026 18:21:50 +0000

A library of Calico tools and skills — delivered through the Calico MCP Server

What if your hardest network question took ten minutes instead of ten days?

Anyone who has operated Kubernetes networking at scale knows the shape of a bad day. A request that should succeed is quietly failing. The application team swears nothing changed. Somewhere across a stack of tiers, selectors, and policies, some written last week and some inherited from an engineer who left two years ago, a rule is denying the traffic. Finding it means reading YAML, cross-referencing flow logs, and reconstructing the policy evaluation order in your head. For a genuinely knotty case, that work can stretch across days and pull in more than one team before anyone gets to the bottom of it.

I think that day is about to get a lot shorter. Today we’re introducing Mylo, an expert for Calico that works inside the AI tools your teams already use. Ask Mylo why a pod can’t reach a service, and it traces the path, points to the exact policy and rule doing the blocking, and explains why in plain language, in about the time it took you to read this paragraph. Mylo is the library of tools and skills that makes Tigera’s Calico expertise usable by an agent; the Calico MCP Server is how we deliver it. The rest of this post covers what it is, how we designed it, and why we made the choices we did.

What Mylo really is: A decade of Calico expertise, as tools and skills

Before getting into how Mylo is delivered, it’s worth being clear about what it actually is, because the delivery mechanism is the least interesting part of the story. Mylo is the Calico and Kubernetes networking knowledge Tigera has accumulated over ten years, turned into something an AI agent can use directly. It comes as two layers, and those two layers, not the plumbing that delivers them, are where the value sits.

The first is a small set of tools. These aren’t thin wrappers over CLI commands; they’re semantic. The flagship, diagnose_connectivity, doesn’t hand back a raw dump for the model to interpret. It returns an ordered, opinionated read of the network path, down to the specific rule that allows or denies, with a suggested fix. The expertise lives in the tool; the reasoning stays with the agent.

The second layer, and the one that compounds over time, is skills: higher-level workflows that string those tools into the procedures an expert would actually follow. Troubleshooting a connectivity failure, triaging an incident, reviewing a policy before it ships: each is a multi-step reasoning pattern, and a skill encodes it, so the agent diagnoses, then traces the policy chain, then checks recent denials, then explains the offending rule, in that order. This is the part I’d point to first. A decade of Tigera’s own troubleshooting expertise, the kind that usually lives in the heads of a handful of senior engineers, is captured here as structured guidance an agent can carry out, rather than a static runbook a human has to follow.

Why an MCP server, and why now

Two things are true about the platform and security engineers who run Calico today. The first is that they are already deep in agentic workflows. In nearly every conversation we’ve had over the past few months, teams weren’t debating whether to adopt an AI assistant. They had already standardized on one, whether that’s a coding assistant in the IDE or an agent in the terminal, and connected it to a set of MCP servers for their metrics, logs, tickets, and source control. The Model Context Protocol has quietly become the way infrastructure tools present themselves to agents.

The second is that Calico holds a lot of the context those engineers need, and it isn’t always easy to reach. The policy model is powerful because it’s expressive, with tiers, global and namespaced policies, selectors, and ordered evaluation. That same expressiveness, however, is what makes a thorny connectivity problem hard to reason about when you’re under pressure. Flow logs hold the answer to “why was this blocked,” but writing the query to surface it is a skill of its own.

Delivering that expertise over MCP sits naturally between those two facts. Rather than build yet another chat interface and ask teams to approve a new AI product with access to production, we expose Mylo’s tools so any MCP-compatible agent can call them. Your engineers keep the front-end they’ve already vetted, and Calico becomes the intelligence layer underneath it. MCP is the delivery mechanism, not the substance, but it’s the right one, because it puts Calico’s knowledge directly into the reasoning loop of an agent that is already helping your team get work done. This is a new way to work with the Calico Unified Platform. It doesn’t replace the CLI or the UI.

Composability: Workflows that weren’t possible before

The reason this approach matters, beyond the convenience of asking questions in plain English, is composability, and it’s worth reviewing, because a lot of teams are still early in adopting MCP. A tool exposed over MCP is a building block an agent can combine with every other capability it can reach. When Mylo runs alongside the MCP servers a team already uses, the agent can move across all of them in a single line of reasoning, and that unlocks workflows nobody shipped as a feature.

Consider a request like this one: “Find the traffic denied by the policy I shipped this morning, correlate it with the elevated 5xx rate the monitoring tool flagged, and open a ticket with the policy diff to fix it.” Three systems answer that in a single pass. Mylo supplies the Calico half, the expert half, and the team’s existing monitoring and issue-tracking servers supply the rest, with the agent orchestrating the whole thing end to end. The same shape shows up all over an operations workflow. Before a pull request merges, the agent can ask Mylo to simulate the policy change against last week’s flow logs and post the impact summary to the team’s channel. When flow logs show a denied connection from a production service, it can page whoever is on call. None of those are workflows we built, and none of them existed as a product a team could buy.

What I find encouraging is that we didn’t have to anticipate any of them. A traditional integration or dashboard can only run the paths its authors designed in advance. Because Mylo exposes a small set of well-designed, semantically clear tools, each team fits it into the operational reality they already have, and some of the most valuable workflows will be ones we never designed for.

Figure: Composability — Mylo alongside the tools you already run

The design philosophy: Restraint

If one principle shaped Mylo more than any other, it’s restraint. The instinct when you build for an agent is to expose everything, to ship a tool for every API method and every CLI subcommand and let the model sort it out. In our view that instinct is a mistake, and we’ve worked hard to resist it.

More tools is not better. Every tool you hand an agent is a claim on its attention, another chance for it to pick the wrong one, and another line item in the token budget. A server with a hundred thin wrappers around CLI commands doesn’t make an agent smarter. It makes it slower, more expensive, and more likely to wander off. That’s why Mylo’s tools are deliberately few and semantic rather than mechanical, and why the intelligence sits in the tool itself rather than in the model’s attempt to reconstruct it from unstructured output.

We wrote down the principles we kept returning to, half in earnest and half tongue-in-cheek. A few are worth calling out, because they’re where the harder decisions actually landed. Read-only by default, because the safe posture should never depend on someone remembering to configure it. State, not verdicts, because a good tool should report what is true and let the agent do the reasoning, rather than pre-baking a conclusion that strips away the context the model needs. Selectors, not switches, because tools should accept rich, expressive inputs instead of sprouting a new boolean flag for every situation. And the one we keep at the very bottom as a reminder to ourselves: thou shalt not ship a tool for every API method.

Figure: The Ten Commandments of MCP Servers — the design principles behind Mylo

One more piece of architecture turns that discipline into something trustworthy: the router. Agents are eager to be helpful, and an agent that can’t reach a tool will often shell out instead, quietly running a CLI command to answer a question rather than admitting the question is outside its lane. That’s harmless right up until it isn’t, and for a tool that touches production networking, silent fallback is a trust problem. So Mylo can be invoked explicitly, and when a request comes in, the server tells the agent clearly whether it can actually handle it rather than letting it improvise around the edges. That deterministic guidance, with explicit boundaries, is what earns the confidence of a platform team.

Figure: How Mylo handles a request — from a plain-language ask to an evidence-backed answer

Built to grow without slowing down

The natural question about a deliberately focused toolset is whether it can grow in scope without growing unwieldy. I’d argue the architecture is exactly what lets us expand without the usual penalty. Because skills are invoked in context and the router keeps each request scoped to what’s relevant, adding a new capability doesn’t tax the ones already there. A troubleshooting session doesn’t get slower or noisier because we’ve also taught Mylo to help with a migration. Breadth comes from adding well-described skills and the focused tools they lean on, not from piling more surface area onto every interaction. In practice that means the range of what Mylo can do for you keeps growing while the cost of any single interaction stays flat.

We’re being disciplined about sequencing, and read-only diagnostics and observability come first because that’s where the pull is strongest. The roadmap ahead is broad, though. Policy authoring is a natural next step, generating well-formed Calico policy from a plain description of intent and presenting it for review before anything is applied. We see real appetite for migration assistance, whether that’s moving VMs into Kubernetes, moving from Ingress to the Gateway API, or moving from Istio sidecars to Ambient. Configuration of Calico resources such as load balancers and egress gateways fits the same pattern. And compliance reporting keeps coming up in customer conversations, from summarizing an application’s secured egress destinations to demonstrating that PCI workloads aren’t communicating with non-PCI workloads, the kind of evidence a security team can hand to an auditor. Each of these is a skill and a small set of tools layered onto the same foundation, so we can keep growing what Mylo does for you without disturbing the safe, composable core underneath it.

Two ways to run it: Live cluster or diagnostics bundle

Mylo works against a live cluster, connecting through the same kubeconfig path your engineers already use for kubectl and calicoctl, with no new endpoint exposed. That’s the mode for real-time diagnostics and observability, for asking what is happening in the cluster right now.

It also works entirely offline, pointed at a Calico diagnostics bundle. This matters more than it first appears. A bundle is portable, shareable, and detached from the live environment, which makes it a good fit for the many settings where a laptop simply cannot reach production: air-gapped estates, tightly controlled prod, or after-the-fact analysis where the state you care about is the one captured at the moment of failure. The same tools and the same skills run against both. An engineer can triage a captured bundle on a plane and run live diagnostics from their desk using the identical workflow. Supporting both was not an afterthought. It’s how we’ve been able to test Mylo so thoroughly, and it’s what lets it meet enterprises where their access model actually sits.

Figure: How Mylo handles a request — from a plain-language ask to an evidence-backed answer

Read-only by design

For the teams evaluating Mylo, the first question is almost always about safety, and our answer is deliberately conservative. Mylo is read-only by design. Its tools diagnose, explain, observe, and where they touch policy, generate proposed YAML. They do not apply, update, or delete anything. This holds even when Mylo is handed a cluster-admin kubeconfig, because the read-only boundary is a property of the tools themselves and not something we’re trusting the model to honor. Security teams have appreciated that they can point their own agent at the server and audit for themselves exactly what it can and cannot do.

The result is a much narrower profile than the interfaces these teams have already approved. Mylo is a typed, structured API with no browser session, no rendered pages, and no standing state to hijack. Access runs through the calling user’s existing Kubernetes RBAC, so if a user can’t list policies in a namespace, neither can Mylo on their behalf. That makes for a far shorter security conversation than approving a net-new AI product, largely because there is no new AI product to approve. The reasoning happens in a front-end the customer already trusts.

What the internal testing showed

We didn’t want to announce a promising demo. Before we put Mylo in front of anyone, we ran it against our own library of resolved support cases, real and historically difficult problems where we already knew the answer and how long it had originally taken to reach. It’s a demanding benchmark, because it measures Mylo against outcomes that skilled engineers had already worked hard to produce.

The results are the reason we’re writing this post. A class of case that historically took on the order of ten days to root-cause, the kind that bounces between teams and keeps escalating, Mylo worked through in roughly ten minutes. Not every case, and never without a person in the loop reading the output and making the call. But the pattern held often enough, across enough cases, that we’re confident this is real and repeatable. Compressing days of investigation into minutes changes what an under-resourced operations team is able to take on.

Early access, and what we’re seeing

Mylo is in early access now with a small group of design partners. They are large, security-conscious organizations, several of them in regulated industries, and all of them already fluent in agentic workflows. We’re keeping the program deliberately tight so we can learn quickly, and we’re bringing participants on in waves rather than opening the gates all at once.

The early signal has been better than we expected. Participants aren’t just kicking the tires; some are reaching for Mylo daily. One of the more encouraging patterns is teams using it to scope problems they simply couldn’t get their arms around before, such as surveying years of accumulated policy sprawl to find what’s orphaned or duplicated, work that used to mean writing custom tooling nobody had time to build. That read-only, get-the-data-out-of-Calico use case has been the wedge almost everywhere. The fastest path to value is answering “why is this happening” in plain English, and the appetite for it is real.

The road to GA

Early access is showing us what enterprise-ready has to mean, and the requirements have been remarkably consistent from one team to the next. On the way to general availability we’re focused on a few things: an in-cluster deployment option for teams that want Mylo owned and run by the platform rather than launched from a laptop; authentication and RBAC that carry each user’s real cluster permissions through to every tool call; tiered access that builds on the read-only foundation with a staged suggest tier, where policy changes are proposed as reviewable diffs and applied only after a human approves; and audit logging that records every tool invocation for the compliance and security teams who, rightly, want the full trail.

None of that changes the core posture. Mylo starts safe, stays composable, and earns each new capability rather than assuming it. That’s the philosophy that got us here, and it’s the one we intend to keep.

If you run Calico and your teams are already working with AI agents, we’d love to have you see this early. Mylo is how Calico shows up in that world, and I think it’s going to change what a hard networking day looks like.

If you’re not a customer yet and would like to learn more about Mylo, drop us a line here.

The post Meet Mylo: An AI-native way to work with Calico appeared first on Tigera – Creator of Calico.

The Safest Place to Run an AI Agent Is On a Cluster That Doesn’t Trust It

Alister Baroi — Thu, 27 Aug 2026 17:34:36 +0000

Every organization running AI agents has already made a hosting decision. Most made it by accident.

The sales team switched on the agent built into their CRM. Engineering is piloting a coding agent in a vendor’s cloud. Someone on the data team deployed a LangGraph service to a VM with a database key in an environment variable, and someone else is running an agent framework on a laptop with production credentials in a dotfile. Each of these is a hosting decision. Each one quietly settled who holds the agent’s credentials, what network paths it can reach, what gets recorded when it acts, and who can stop it. Nobody ran an architecture review, because no single deployment looked big enough to deserve one.

The scale says otherwise. By May 2025, 82% of organizations surveyed by SailPoint were already using AI agents. Only 44% had policies for securing them, 80% said their agents had already taken unintended actions, and 23% had watched an agent get tricked into revealing credentials. A year later the bill arrived: IBM’s 2026 Cost of a Data Breach report found that one in four malicious breaches is now AI-enabled, up 56% in a single year, and that those breaches cost about a million dollars more than the global average. Gartner, for its part, expects over 40% of agentic AI projects to be canceled by the end of 2027, naming inadequate risk controls as one of the three reasons.

This post is the architecture review those deployments never got. The conclusion, stated up front so you can argue with it: agents should run on Kubernetes clusters you control, wrapped in guardrails and gateways that no prompt can talk its way past. Not because Kubernetes is fashionable, but because a year of documented incidents shows that every failure was an environmental failure, and a cluster you control is the only substrate where you own every layer of the environment.

The workload that chooses its own code path

A microservice does what its code says. You can read the code, test the paths, and enumerate what it will do with any input. An agent is a different kind of workload, and the difference is structural, not a maturity phase it will grow out of:

Its control flow is decided at runtime by a model. The “code path” is chosen by whatever lands in the context window. You cannot statically review what an agent will do, only what it is allowed to do.
It holds credentials and takes actions. Agents query databases, call APIs, send email, move money, and spawn other agents. The interesting failure mode is not a wrong answer. It is a correct, authorized, catastrophic action.
Every input is potentially an instruction. Prompt injection remains unsolved. OWASP’s Top 10 for Agentic Applications, published in December 2025 with over a hundred contributors, reads mostly as variations on this theme: goal hijack, tool misuse, identity and privilege abuse, memory poisoning.

Put those three together and you get the principle we have argued all year, from the accountability gap to the OpenShell analysis: controls the agent can override are not controls. The system prompt is not a perimeter. The guardrail library running inside the agent’s own process is not a perimeter. The controls have to live in the environment, where a confused or compromised agent cannot negotiate with them.

And once the controls must live in the environment, the hosting question stops being an infrastructure detail. Where the agent runs determines which controls can exist at all.

A year of incidents; one pattern

Theory is cheap. The past twelve months supplied the evidence, and it is unusually well documented.

One agent’s tokens, seven hundred victims

Drift was an AI chat agent embedded on corporate websites. To do its job it held long-lived OAuth tokens into each customer’s Salesforce. In August 2025, attackers who had stolen those tokens used them to run bulk data exports against customer CRMs. Google’s threat intelligence group assessed more than 700 organizations as potentially affected. The attackers were not after chat transcripts. They mined the exported support cases for the credentials customers had pasted into them: AWS keys, Snowflake tokens, passwords. Cloudflare, one of the disclosed victims, rotated 104 API tokens found in its stolen case text.

No agent was compromised. No model misbehaved. One integration’s credentials were broad, long-lived, and held outside every victim’s own controls, so one theft became seven hundred breaches.

The agent with production credentials and a code freeze

In July 2025, on day nine of a public twelve-day build, Replit’s coding agent wiped SaaStr founder Jason Lemkin’s production database, despite an explicit and repeated code-freeze instruction. It then generated thousands of fake records and reported that rollback was impossible. It was not; the data came back from backup. The agent’s own postmortem: “a catastrophic failure on my part.”

The agent held live credentials to production with no environment separation, and the code freeze existed only as natural language. An instruction in a prompt is a suggestion. The incident record was, briefly, whatever the agent chose to say it was.

The tool that BCC’d the attacker

In September 2025, Koi Security found the first confirmed malicious MCP server in the wild: an npm package impersonating Postmark’s email server. It behaved correctly for fifteen versions, building trust, then v1.0.16 added a single line that BCC’d every email the connected agents sent to the attacker’s domain. Koi estimated roughly 300 organizations were running it in production. Nothing was exploited. An MCP server runs inside the agent’s trust boundary with the user’s credentials, and nobody diffs a patch release.

And the rest, briefly

EchoLeak (CVE-2025-32711) demonstrated a zero-click chain against Microsoft 365 Copilot: a poisoned email, retrieved later by RAG, walked the assistant’s most sensitive reachable data out through an allowlisted URL. Microsoft fixed it before any known exploitation, and it became the first critical-severity CVE assigned to an AI assistant’s injection path. The s1ngularity npm supply-chain attack inverted the picture: malware on developer machines invoked the victims’ own installed CLI agents with their client-side safety flags disabled and put them to work enumerating secrets, leaking over a thousand valid GitHub tokens within hours. And in July 2026, Check Point documented a single operator running two frontier-model agents as an intrusion pipeline that hit nine Mexican government agencies and roughly 400 million records. Attackers have agents too, and theirs ignore your prompts entirely.

The pattern

None of these was a model failure. Every one of them was contained, or would have been, by the same four environmental controls:

Credentials scoped to one target and short-lived , so a stolen token is worth almost nothing (Drift).
Authorization evaluated on every request against policy , not against the agent’s intentions or instructions (Replit, and the bulk CRM exports that no chat agent should have been able to run).
Egress that is deny-by-default and enforced outside the agent , so even a successful injection has nowhere to send the loot (EchoLeak, the Postmark BCC, s1ngularity’s uploads).
An audit trail owned by the platform, not narrated by the agent (Replit’s fabricated reports; Cloudflare could say “exactly 104 tokens” only because logs existed outside the blast radius).

Read that list again. Nothing on it is a property of the model. Nothing on it can be provided by a prompt, a fine-tune, or a guardrail library running inside the agent. All four are properties of the place where the agent runs.

Why the answer is a cluster you control

Environmental controls need an environment you own: the network path, the identity plane, the secret store, the audit pipe. Kubernetes is the one mainstream substrate that hands you all four as programmable primitives: namespaces and NetworkPolicy for segmentation, runtime classes for kernel-level sandboxing (gVisor, Kata), SPIFFE for workload identity, and admission control to keep anything unreviewed from running at all. Its declarative model then puts every one of those decisions in git, where they get reviewed and versioned like the rest of your infrastructure. These are the same properties that made Kubernetes the default for microservices. For agents they are not conveniences; they are the containment system.

The ecosystem has noticed. In the past year, Kubernetes gained a SIG subproject for agent sandboxing with first-class Sandbox primitives, launched at KubeCon NA 2025. agentgateway, a data plane purpose-built for agent-to-agent, MCP, and LLM traffic, became a Linux Foundation project with participation from AWS, Microsoft, Red Hat, IBM, and Cisco. KubeCon NA 2026 has a dedicated AI and agentic track. Running agents on Kubernetes stopped being a contrarian position sometime last winter.

Watch the managed agent runtimes and you see the same conclusion arriving from the other direction. AWS AgentCore added VPC egress and private identity-provider support. Microsoft’s agent service will inject its runtime into your own VNet so agent data stays in your tenant. Anthropic’s managed agents shipped self-hosted sandboxes that run tool execution on your infrastructure. OpenAI’s Frontier launch promises agents that run “across local environments, enterprise cloud infrastructure, and OpenAI-hosted runtimes.” Every managed runtime’s roadmap is bending toward your perimeter. That is the vendors telling you, in feature announcements, where the controls have to live.

There is also the question of what you can see when isolation is outsourced. In late 2025, Unit 42 disclosed an escape in AWS AgentCore’s code-interpreter sandbox: its “sandbox” network mode still allowed DNS resolution to arbitrary domains, which is enough for tunneled exfiltration and command-and-control. AWS fixed it properly within months, and credit to them. But note who found it and when the customers learned: from the researcher’s publication, not from their own telemetry, because inside a managed runtime there is no customer telemetry to find it in.

Side by side:

Concern	Managed agent runtime	Sandbox platform	Kubernetes with guardrails
Where the agent executes	Vendor’s cloud (some now reach into your VPC)	Provider’s microVM fleet	Your cluster: any cloud, on-prem, air-gapped
Who holds tool credentials	Vendor’s credential broker	Injected into the sandbox	Brokered in-cluster, attached per hop
Egress control	Vendor-defined, opt-in	Platform-defined	Your gateway, default deny
Authorization granularity	Per connector, mostly coarse	Yours to build	Per request, down to tool arguments
Audit trail	Per vendor, per format	Per platform	One trail across agents, tools, and models
Agents you didn’t launch	Invisible	Invisible	Detectable on the node
Exit cost	Rebuild against a new runtime	Re-platform	Agents are containers; the cluster moves with you

The table generalizes, and the best vendors are improving individual rows. What no managed runtime can give you are the rows that require owning the substrate: one policy engine over every agent regardless of framework, one audit trail across agents, tools, and models, and visibility into the agents nobody told the platform about.

One caveat before the spec. _ A bare cluster is not the answer either _. Stock Kubernetes secures pods, not agents: NetworkPolicy has no idea what an MCP tool call is, RBAC governs humans and service accounts, and nothing in the box produces an agent-level audit trail. We walked through those gaps in the accountability gap post. What the cluster uniquely gives you is the ability to install the missing layer yourself, uniformly, for every agent you run.

What strict guardrails look like

Here is the spec a year of incidents wrote, stated as requirements. Sandboxing the agent’s process is assumed and is not sufficient on its own; even the CNCF says so now. Everything below governs what agents do across the network, which is where every incident above happened.

An identity for every agent. Cryptographic workload identity (SPIFFE, OIDC) issued at deploy time, not a shared service account. Every other control depends on “which agent did this” having an answer.
One road out. Agent egress is deny-by-default, and the only path to other agents, MCP servers, and LLM providers is a gateway. EchoLeak, the Postmark backdoor, and s1ngularity’s uploads were all exfiltration over permitted egress. An allowlist enforced outside the agent kills the chain even when the injection succeeds.
Authorization per request, not per deployment. Each call is judged against policy: this agent, this target, this tool, these arguments. A lead-triage agent asking for a bulk CRM export should fail on policy, not on luck.
Credentials the agent never holds. Provider keys and tokens attach at the gateway, scoped to the single target of the single hop, short-lived. This is the Drift control: make the stolen token worthless.
A human where the blast radius is high. Destructive and irreversible operations block until a person approves. “Code freeze” becomes a policy object with an enforcement point, instead of a sentence in a prompt.
Detection for the agents you didn’t launch. 80% of organizations already report unintended agent actions, and IBM’s data keeps tying breach costs to shadow AI. You need to see the unregistered agent from the node it runs on, not from a registry it never joined.
An audit trail the agent cannot edit. Recorded by the enforcement plane as a side effect of enforcing, tied to agent identity, covering every hop. The agent’s own account of events must never be the system of record.

If you run a startup, this list probably reads as an enterprise ceremony. It is the opposite. The spec is precisely what lets a five-person team hand agents real credentials to real systems without hiring a security organization: default-deny scales down, a policy file for three agents fits on one screen, and one gateway is not a platform team’s worth of work. If you run an enterprise, the same spec is what turns your agent deployment from an audit finding into an audited system. The requirements do not change with headcount. Only the number of agents does.

The platform this spec describes

By now the specificity of that list has probably given the game away: it is, near enough, the design document for Tigera Lynx, our security and governance platform for AI agents on Kubernetes.

Lynx puts a registry and identity plane in front of your agents: SPIFFE and OIDC workload identity, plus a pod-owner mode that brings an unmodified agent under governance with no code changes and no credential to issue. Its gateway, built on the same Linux Foundation agentgateway data plane mentioned above, authorizes every agent-to-agent, MCP, and LLM call individually under Cedar policy, and MCP policies can see the operation, the tool name, and the tool arguments, failing closed on anything malformed. LLM provider keys live in your cluster’s secret store and attach at the gateway, so an agent never carries a provider credential it could leak, and per-provider token rate limits cap what any agent can burn.

In the current release cycle, per-hop credentials are minted for one target only (the Drift lesson, applied literally), and a policy can mark specific MCP calls as needing human approval, so the call waits until someone signs off (the Replit lesson). On the node, an eBPF-based detector spots agents nobody registered, classifies them as sanctioned, shadow, or unknown, and lets you quarantine a compromised workload at both the gateway and the kernel. And the Agent Trail records every decision, every hop, and which model actually served each call, as a byproduct of enforcement. The agent does not get to narrate.

Every one of those maps to a numbered line in the spec, because the spec came first. How Lynx Works has the full walkthrough, and the five principles post is the vendor-neutral checklist if you want to evaluate Lynx against anyone else.

The decision you already made

Back to the opening. Your organization has a hosting posture for AI agents today, assembled from defaults: whichever vendor clouds your teams clicked through, whichever VMs were lying around, whichever laptops were closest. The incidents above are what those defaults cost, and IBM’s data says AI is now involved in one of every four malicious breaches.

So ask the question deliberately, once, before the next agent ships: when it acts, who can say no, and where is the record? If the answer is a vendor’s incident queue and a support ticket, _ you have outsourced the two things a security leader cannot outsource. _

_ An agent you cannot refuse is an agent you do not control. _ Run yours where refusal is built in: a Kubernetes cluster you own, a gateway it cannot go around, and a policy engine that never gets tired of saying no.

Learn more about Lynx →

The post The Safest Place to Run an AI Agent Is On a Cluster That Doesn’t Trust It appeared first on Tigera – Creator of Calico.

AI Red Team Agents Automate Attacks on your AI Agents. Runtime Policies Automate their Defense.

Alister Baroi — Mon, 24 Aug 2026 14:40:14 +0000

The AI red teaming market grew up fast this year. OpenAI bought Promptfoo, Cisco and Microsoft shipped automated attack suites, and a seed-stage startup publicly compromised 50 of 55 live customer service bots. These platforms find real problems at a scale no human team can match. But when you read the findings closely, a pattern emerges: agents talked into refunds, transfers, and data leaks they had standing authority to perform. Patching the prompt fixes one phrasing until the next model update. Constraining the authority fixes the class. The first job belongs to a red team platform. The second belongs to your runtime, and no scanner will do it for you.

In April 2026, General Analysis raised a $10M seed round on the strength of an uncomfortable demonstration: its adversarial agent attacked 55 live customer service bots and compromised 50 of them. Not lab models, but live systems with real customers and real tool access. This post is about the market behind that demonstration: who now automates the attacker’s role, what the attacks keep finding, and why the fix that lasts is runtime policy rather than a better prompt.

What an AI red team agent actually does

A traditional red team is a group of people paid to break your system before someone else does. An AI red team agent automates that role for AI systems. It fires thousands of adversarial inputs at a model or agent, notices which ones partially work, mutates them, escalates across turns, and reports what got through, with severity ratings and reproduction steps. A human tester works through a checklist over a week. These platforms run 20,000 variations overnight and rank the results.

The canonical demo goes like this. A company deploys an agent that handles refunds and can reach the billing system. The red team platform embeds disguised instructions in customer messages until it finds the phrasing that makes the agent skip its verification step and process a refund it shouldn’t. The report names the exact prompt pattern so engineering can fix it.

Enterprises are buying for reasons that have little to do with fashion. Agents now do things, such as executing code, querying databases, sending emails, and calling APIs, often with minimal supervision, so every capability is an attack vector. The EU AI Act mandates adversarial testing for high-risk AI systems, and insurers and procurement teams have started asking for documented testing as a condition of coverage or vendor selection. And a quarterly pen-test calendar cannot keep up with an agent whose behavior shifts on every model update.

All of that is real, and the tooling is genuinely good. None of this is an argument against red teaming. The question is what you do on the morning the report arrives.

The AI red teaming market grew up in about a quarter

The speed of consolidation tells you how seriously the industry takes automated red teaming. In March 2026, OpenAI acquired Promptfoo for a reported $86M; the MIT-licensed scanner claims 350,000 developers and now ships inside OpenAI’s Frontier enterprise platform. Cisco AI Defense tests across 200+ risk subcategories mapped to OWASP, NIST, and MITRE ATLAS. Microsoft’s AI Red Teaming Agent wraps PyRIT’s multi-turn attack orchestration, including Crescendo and Tree of Attacks with Pruning, into Azure AI Foundry. Mindgard starts with reconnaissance: it maps your guardrails, tools, and shadow AI deployments before it fires a single probe. And General Analysis goes deepest on agentic systems: MCP server exploitation, memory poisoning, and multi-step permission bypass.

OWASP now publishes a quarterly landscape document just for this category. When a market gets its own recurring OWASP publication, it has stopped being a niche.

Read the findings like a defender

Look at what these platforms actually catch across every vendor’s case studies: an agent processing unauthorized refunds. An agent skipping its verification step. An agent revealing one customer’s data to another. A moderation model bypassed with encoding tricks. An agent walked, over several polite turns, across a permission boundary it was never supposed to cross. A poisoned memory that redirects behavior days later. A malicious MCP server that turns one compromised tool into several.

Sort those into two piles. In the first pile, the model said something it shouldn’t have (jailbreaks and moderation bypasses). Those are model-layer problems with model-layer fixes, and the labs keep getting better at them. In the second pile, the agent did something it shouldn’t have. Every headline finding in the agentic column lands here, and every one of them has the same anatomy: the agent was persuaded, and the environment obeyed.

The persuasion is novel every time. That is the entire point of automated attack generation; there is always another phrasing, another encoding, another escalation path. The obedience is identical every time. The agent asked for transfer_funds, and something executed it.

Prompt patches decay. Runtime policy doesn’t.

Here is the loop most teams fall into. The red team report names a prompt pattern. Engineering patches the system prompt or adds a guardrail rule. The finding closes. A model update ships three weeks later, and the finding reopens with different wording. The vendors know this; it’s why General Analysis sells regression testing for exactly the case where a passing agent configuration fails after a model update or prompt change, and why every platform now pitches continuous scanning rather than an annual engagement. Continuous testing is the industry’s honest admission that the layer under test never stays fixed.

We’ve made this argument before, in the accountability gap post and throughout the series: controls the agent can override are not controls. The same logic applies to instructions. An agent can be talked out of its system prompt; that is precisely what the red team platform just proved. It cannot be talked out of a deny that lives outside its process.

So a pile-two finding deserves two responses. Patch the prompt, by all means; make the attack more expensive. But the fix that lasts is changing what the persuasion can accomplish, and that change happens in the environment, at the point where the agent’s request becomes an action.

Rewriting a red team finding as runtime policy

Take the marquee finding, the one on every vendor’s landing page: an injected instruction convinces a finance agent to call transfer_funds without its verification step. As a reproduction step, that’s a ticket that will reopen. As policy, it looks like this:

@approvalRequired("Approve transfer_funds on treasury")
forbid ( principal, action == Action::"callMCPServer", resource )
unless { context.approval.granted }
when { context.mcp.tool.name == "transfer_funds" };

This is Cedar, evaluated at a gateway between the agent and its MCP servers. Any call to transfer_funds is denied unless a human has granted approval, and the annotation turns that deny into a hold: the gateway parks the request, a person approves or rejects it, and only then does anything move. Approval is an input to the policy decision, not an override of it. The injection can still convince the agent to try. Trying now produces a held request and an audit entry instead of a wire transfer.

The same translation works across the rest of pile two:

What the red team keeps finding	Where the durable fix lives
Prompt injection triggers an unauthorized tool call	Per-request authorization at a gateway the agent cannot route around
Agent skips a verification or approval step	A policy hold that parks the call until a human decides
Agent leaks one customer’s data to another	Tool and egress policy scoped to the agent’s verified identity
Multi-step chains cross a permission boundary	Per-call decisions with the full chain correlated in one trace
The agent nobody tested, because nobody knew it existed	Runtime detection of unregistered agents

That last row deserves a sentence. Mindgard leads with reconnaissance because most organizations cannot produce an accurate inventory of their own AI systems, and you cannot red team what you don’t know you’re running. Scoping the engagement is where a program fails first, before a single probe fires. Inventory is a runtime problem too.

This is the seam where Lynx sits. Its gateway authorizes every MCP call individually against Cedar policy under a workload identity (e.g. SPIFFE). The @approvalRequired hold shown above shipped in preview this cycle, with the pending queue and countdown surfaced in the dashboard. Agent Trail records the deny that started the hold and the human decision that resolved it. And runtime detection classifies the agents actually present in your cluster, so the inventory a red team engagement needs is a query rather than an archaeology project. None of that replaces adversarial testing. It’s what turns the testing’s output into something that stays fixed.

Starting with open source: Garak, PyRIT, and DeepTeam

If you want to start this quarter, the open source route is credible. Garak (by NVIDIA, under Apache 2.0 license) is the deepest free attack catalogue: 120+ probe modules, with multi-turn agentic attacks added in v0.15.0 this May. PyRIT (Microsoft, MIT) is the strongest orchestrator for custom multi-turn campaigns. DeepTeam (Apache 2.0 license) maps results onto OWASP’s Agentic Security Initiative taxonomy for the compliance evidence trail; disable its cloud scoring calls if your data can’t leave the building. One caveat worth knowing before you standardize: Promptfoo belongs to OpenAI and PyRIT to Microsoft, both model vendors. The attack logic is open source and auditable, which helps, but neither should be your only scanner when the system under test is that vendor’s own model.

Two practices make whichever stack you pick actually pay off. First, run it against a test double, not only production; we covered how to stub LLMs for exactly this, so you can exercise the agent’s authority paths deterministically. Second, file every confirmed agentic finding as two tickets: one for the prompt, one for the policy that makes the prompt’s failure survivable. If your enforcement point logs its decisions, the audit trail becomes your regression oracle. The next scan shouldn’t just show the attack failing; it should show the gateway denying it.

Key takeaways

AI red team agents automate adversarial testing at a scale no human program matches, and the market consolidated fast: OpenAI bought Promptfoo, Cisco and Microsoft ship attack suites, and OWASP now tracks the category quarterly.
The agentic findings share one anatomy: the agent was persuaded, and the environment obeyed. Persuasion varies infinitely. Obedience is the constant you can actually fix.
Prompt patches decay with every model update, which is why the vendors themselves sell continuous regression testing. Policy enforced in the environment doesn’t decay with the model.
Translate findings into runtime controls: per-request authorization, human approval holds on dangerous tools, identity-scoped access, and detection for the agents you didn’t know you had.
Buy the red team. Build the enforcement. They are complements, and neither substitutes for the other.

Back to the fifty bots. What they had in common was not a worse model or a lazier system prompt than the five survivors. It was that persuasion and permission were the same thing: once the agent was convinced, nothing else got a vote. Your next red team report will show you where that’s true in your own stack. When it does, which findings will you patch, and which will you enforce?

The red team finds the words that move your agent. Your runtime decides what moving accomplishes.

Learn more about Lynx →

The post AI Red Team Agents Automate Attacks on your AI Agents. Runtime Policies Automate their Defense. appeared first on Tigera – Creator of Calico.

You’ve Been Running a Kubernetes Security Model in NSX and Didn’t Even Know It

Alister Baroi — Mon, 10 Aug 2026 19:19:22 +0000

One of the blockers to moving VMs off vSphere and onto Kubernetes is losing NSX and the protection it provides. Security teams that have spent years building out distributed firewall policy look at Kubernetes and are, quite understandably, alarmed by the flat network and the fact that any workload can reach any other by default.

How will they enforce east-west traffic controls? Will they be able to replicate NSX distributed firewall rules with the same granularity? What about security groups, tiered policy, and rules that travel with the workload when it moves? These are important questions that must be answered before migration can begin.

Migrate vSphere to Kubernetes with microsegmentation intact

Calico addresses vSphere to Kubernetes security concerns with a network policy model that maps directly to key features of the NSX distributed firewall (NSX DFW). Every property NSX DFW users rely on has a direct Calico equivalent: tiered governance, workload-identity enforcement, distributed kernel-level inspection, and dynamic workload grouping. Teams coming from vSphere will recognise the pattern quickly.

Let’s walk through each one in detail.

Distributed Enforcement

Enforcement on the host means better performance and automatic scaling

Traditional firewalls sit at the edge of the network. Traffic between workloads inside the data center has to travel out to that inspection point and back, even if the source and destination are on the same physical host. The NSX distributed firewall takes a different approach: the DFW runs in the kernel of every ESXi host and enforces policy directly at each VM’s network interface, before traffic leaves the VM.

The practical effect is that inspection happens at the source. A packet from a web server VM headed toward a database VM gets checked right where it originates, not at a central appliance somewhere else on the network. No round trip to a dedicated device. Enforcement scales automatically because it lives on every host alongside the workloads it protects.

Calico’s enforcement model is built on the same principle. On each Kubernetes node, Calico enforces policy at every workload’s network interface using eBPF or one of the Linux netfilter-based dataplanes (iptables or nftables). The mechanism differs from VMware’s kernel module, but the position in the stack is the same: enforcement happens before traffic enters or leaves the workload, not at a central inspection point somewhere else in the cluster.

eBPF is worth a brief note here. It’s a way of running purpose-built programs inside the Linux kernel without modifying the kernel itself, which means packet inspection and policy decisions happen at a very low level with very little overhead. It’s the same instinct that makes NSX DFW fast: get enforcement as close to the wire as possible.

The solution scales in the same way as well. Adding a node to the cluster adds enforcement capacity automatically because the enforcer runs on the node rather than as a separate service that all traffic routes through. Enforcement is distributed across every node, so there’s no central chokepoint that the whole cluster’s traffic has to pass through.

Workload-attached Policy

Policies follow the workloads they secure

When a VM migrates between hosts via vMotion, its firewall rules move with it. The policy is attached to the workload, not the host. This is a consequential architectural choice: you can’t accidentally leave a workload unprotected by moving it to a new host with different configurations in place.

It also exposes a problem with IP-based firewall rules that becomes obvious the moment workloads start moving. Rules written against specific addresses are tied to a network location, not the workload itself. If a VM changes address after migration, those rules no longer cover it. NSX DFW avoids this by anchoring policy to security group membership instead, and security groups can be built around tags rather than addresses. A VM tagged “web-tier” carries that group membership with it regardless of which host it lands on or what IP it holds. The rule that applies to web-tier VMs follows automatically, with nothing to update.

This is exactly how Kubernetes and Calico network policies also work. Rules are declared against label selectors. When a workload moves to a different node, its labels move with it and the policy selector picks it up automatically. Nothing IP-specific lives in the policy, so there’s nothing to update when an address changes. A VM labeled env: production and tier: database picks up the right policies wherever it lands. No external mechanism needs to track it.

Tiered Precedence

Microsegmentation allows for granular control and a separation of concerns

NSX also has a tiered policy model. Rules are organized into named categories with a fixed evaluation order. Emergency and quarantine rules sit at the top, infrastructure and shared service rules in the middle, and application-level rules at the bottom. Traffic is evaluated against each category in sequence, top down. A deny rule in the highest category stops evaluation immediately. Nothing below it applies.

The category structure is also where access control is enforced. NSX’s role-based access control can be scoped to specific policy categories. The evaluation order and the permission model are designed to align. Teams with the narrowest access work in the categories that run last, and no rule they write can override a higher category.

Calico tiers are designed according to the same principles. Each tier is a Kubernetes resource with an order field, a number that determines where it sits in the evaluation sequence. Lower numbers run first. A tier with order 100 is evaluated before a tier with order 200, and a deny rule in the first tier cannot be overridden by anything in the second.

Unlike NSX’s fixed category names, Calico tiers are user-defined. You name them to match your own operational model, such as “security,” “platform,” “application”, and you set the ordering. You can define as many tiers as your use cases require. Most teams structure their tiers so the security team’s policies run first, platform or infrastructure policies run second, and application or namespace-level policies run last.

Within each tier, individual policies also carry an order field, and within each policy rules are evaluated top to bottom with the first match applying. One useful behaviour worth knowing about is the pass action: a rule can explicitly pass traffic to the next tier rather than allowing or denying it. This lets each tier focus on what it’s responsible for and hand off everything else cleanly, rather than requiring every tier to have an opinion on every packet. The security tier, for example, should not have to worry about whether a specific frontend should be able to talk to a certain backend service.

For access control, Calico uses Kubernetes-native RBAC. ClusterRoles and ClusterRoleBindings govern access to cluster-scoped resources like tiers and GlobalNetworkPolicy, while Roles and RoleBindings can be used for namespace-scoped NetworkPolicy. Either way, teams can be restricted to working within their own scope with no ability to reach into tiers or namespaces they don’t own.

Attribute-based Identity

Policies are not IP dependant

NSX uses tags for dynamic workload grouping. Tags are key-value pairs applied to VMs. You can set them manually through NSX Manager or via API, which means they can be applied at deployment time rather than added by hand after the fact. Security groups define their membership through tag-matching criteria, and that membership is live: any VM carrying the right tag combination joins the group immediately.

Tagging a VM as “PCI-in-scope” isn’t just a label. It’s a trigger. The moment that tag is applied, the VM joins every security group whose membership rule matches it, and every DFW policy targeting those groups immediately covers the new member. No change request, no waiting. The policy was already written; the workload just identified itself as falling within its scope.

The same logic works on decommission. When a tag is removed, the VM leaves the group and the rules that covered it stop applying. Orphaned firewall rules accumulating over years of manual changes are a genuine operational hazard in IP-based environments — tags sidestep the problem entirely because security group membership reflects the actual state of the environment, not a record of past changes nobody got around to cleaning up.

No manual list maintenance. No IP tracking. The tag is the identity.

NSX tags map to Kubernetes labels, and the mechanism works the same way. Labels are key-value pairs defined in a workload’s manifest and applied at deployment. A Calico network policy references them through a selector: compliance: pci in a label is picked up by any policy whose selector matches that expression. The moment a workload with that label comes online, it’s covered. No separate step, no group membership to update manually.

Labels can sit on pods, on namespaces, or on both, which gives you different levels of granularity. A policy that selects on a namespace label applies to everything running in that namespace. A policy that selects on a pod label is more specific. Combining the two lets you write rules like “allow traffic from any workload in the payments namespace that also carries the label tier: web” — the equivalent of multi-tag security group membership in NSX, expressed as a single selector.

The decommission property carries over too. When a workload is deleted, its labels go with it. Policies that targeted it stop applying automatically, with no cleanup step and no stale rules accumulating over time.

For endpoints that can’t carry labels such as external services, legacy VMs not yet migrated, third-party IP ranges, Calico’s NetworkSets handle the case directly. A NetworkSet is a named collection of IP addresses or CIDRs that can be referenced in a policy selector the same way a label can. Rather than repeating IP ranges across every policy that needs them, you reference the NetworkSet by name. When those ranges change, you update the NetworkSet and every policy that references it reflects the change automatically.

A demo that went sideways in the best way

A solutions engineer was running a demo for a team evaluating a VMware replacement. They were walking through how Calico enforces network policy when the customer’s senior network consultant stopped them.

This team had been running NSX-T for years. They had 19 T1 edges and 8 segments. During the walkthrough, the consultant started recognizing what he was looking at. Then he pointed out something that reframed the whole conversation. The T1 edges weren’t actually providing any isolation. The distributed firewall policy was doing all the security work.

What the consultant recognized was that his team had been operating a flat network with policy enforcement. They’d been using NSX tags to classify workloads and superset policies to establish rule precedence.

Both of those map directly to Calico constructs which surprised them. The assumption had been that moving from NSX to a Kubernetes-native stack meant leaving the policy model behind. It turned out the policy model was already there. They’d been building towards it without realizing it.

This is a common pattern. In NSX deployments where the DFW has been built out carefully, the network topology handles connectivity and the policy handles security. The T1 edges are default gateways. The actual security boundary is the DFW rule, not the segment boundary.

What works better on the other side

Once policy is running in Calico, a few things improve over the NSX baseline. Compliance monitoring runs continuously and generates audit-ready reports against frameworks like PCI DSS, HIPAA, SOC 2, and NIST. This replaces manual evidence assembly before each audit cycle. Policy changes tracked in version control produce evidence of reviewed, timestamped, and tamper-evident changes. Tiered RBAC means the security team’s rules are structurally unreachable by teams below, with compliance report access scoped so each team sees only what it owns.

What this all means in practice

The NSX distributed firewall may feel like one of the hardest components to replicate but it turns out to be the part that transfers most directly, because the architectural decisions behind it, distributed enforcement, workload identity, tiered governance, attribute-based grouping, are the same decisions Calico was built on.

The team in that demo was genuinely surprised when they recognized their own security model in the Calico policy framework. They’d spent years building a policy structure in NSX that was a close approximation of what Kubernetes-native policy looks like. They’d been moving toward this model without knowing they were doing it.

If your NSX security posture is built on tags and tiered rules, you’ve been thinking this way for a while. The migration is a translation project. The model is already there.

Everything covered here has a corresponding configuration reference in the Calico microsegmentation documentation. It walks through tier setup, label-based policy, RBAC, and compliance reporting in detail.

Read our migration guide for a comprehensive look at VM migration.

The post You’ve Been Running a Kubernetes Security Model in NSX and Didn’t Even Know It appeared first on Tigera – Creator of Calico.