DEV Community: Dan Henderson

Cloud Run vs. GKE Autopilot: A Comparison for ADK Agents

Dan Henderson — Fri, 31 Jul 2026 13:00:30 +0000

Cloud Run and GKE Autopilot are both reasonable places to run an ADK agent, and that's exactly why I wanted to put them side by side rather than take either one on faith. Cloud Run is the serverless option: point it at a container and it scales with request volume without you touching a node, a cluster, or a networking layer. In my experience, that's the shape most ADK agents actually take: mostly idle, with an occasional burst of requests, which is exactly what Cloud Run is built for. GKE Autopilot is the other end of the same idea: still hands-off on node management, Google runs the control plane and provisions capacity for you, but it's still Kubernetes underneath, which means the platform-level controls (RuntimeClasses, GPU node pools) that Cloud Run doesn't expose are still there if an agent workload actually needs them.

Both are plausible defaults, and I hadn't actually run one against the other before, so I built the same ADK agent container and deployed it unchanged to both. Same image, cold starts measured on both across repeated runs, one real load test pattern rather than the three I'd originally planned (I'll explain that scope cut honestly further down). Nothing here is invented, rounded, or smoothed over. This is what actually happened when I ran it.

Architecture

Figure 1: Same image, same destination API, different identity path. The Workload Identity binding on the GKE side is not optional, and nothing on the Cloud Run side needed it.

One container image, built once, pushed to Artifact Registry, deployed unchanged to both platforms:

Cloud Run: min-instances=0, autoscaling, default Compute service account, already holding roles/aiplatform.user.
GKE Autopilot: same image, a Deployment I manually scaled between 0 and 1 replicas to simulate idle/cold-start behavior (Autopilot has no native scale-to-zero; I'll say that plainly here and again later, because it matters), plus a RuntimeClass attaching gVisor sandboxing, which is really the whole reason this build exists.

Both were deliberately configured to sit at zero when idle, not because that's how you'd necessarily run either in production, but because a cold-start comparison only means anything if both platforms are actually starting from cold. min-instances=0 on Cloud Run and scaling the GKE Deployment down to zero replicas are the two mechanisms that make that possible, and they're the reason the numbers in the Cold-Start Comparison section exist at all.

Walkthrough

Deployed side by side, so the equivalent step on each platform sits next to its counterpart rather than buried in two separate sections.

Shared: one image, one Artifact Registry repo

gcloud builds submit --tag europe-west2-docker.pkg.dev/PROJECT_ID/adk-agent-repo/adk-agent:latest

Figure 2: One image, one repo, deployed unchanged to both platforms below.

Cloud Run

gcloud run deploy adk-agent-cloudrun \
  --image=europe-west2-docker.pkg.dev/PROJECT_ID/adk-agent-repo/adk-agent:latest \
  --region=europe-west2 \
  --min-instances=0 \
  --max-instances=3

Figure 3: Deployed, scale-to-zero by default, no identity configuration step required.

No identity step here, none. The service ran as the default Compute service account, which already held roles/aiplatform.user from earlier testing, and calls to Vertex AI just worked.

GKE Autopilot

gcloud container clusters create-auto adk-agent-cluster \
  --region=europe-west2

Figure 4a: Cluster provisioned.

Figure 4b: Confirmed running in Autopilot mode.

kubectl apply -f deployment.yaml

Figure 5: Service endpoint live, deployment deliberately scaled to zero: the manual stand-in for scale-to-zero used throughout this build.

The Workload Identity gap

This is the bit I want to slow down on, because it's the clearest evidence for the whole point of this article. Every single request to Vertex AI from the GKE pod failed:

403 PermissionDenied: aiplatform.endpoints.predict

Worth checking rather than assuming: confirm roles/aiplatform.user is actually bound to the Compute service account before going further.

gcloud projects get-iam-policy PROJECT_ID \
  --flatten="bindings[].members" \
  --filter="bindings.role:roles/aiplatform.user" \
  --format="table(bindings.members)"

It was. The Compute service account already had roles/aiplatform.user, the exact identity Cloud Run had just used successfully. GKE Autopilot enforces Workload Identity by default, and a pod does not automatically inherit the node's service account permissions the way Cloud Run's runtime does. Fixing it took three explicit steps, none of which have a Cloud Run equivalent:

kubectl create serviceaccount adk-agent-ksa

gcloud iam service-accounts add-iam-policy-binding \
  PROJECT_ID-compute@developer.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:PROJECT_ID.svc.id.goog[default/adk-agent-ksa]"

kubectl annotate serviceaccount adk-agent-ksa \
  iam.gke.io/gcp-service-account=PROJECT_ID-compute@developer.gserviceaccount.com

Then serviceAccountName: adk-agent-ksa on the pod spec, and the same identity Cloud Run got for free finally worked on GKE too. This is a real, non-trivial cost of choosing GKE Autopilot.¹

Cold-Start Comparison

Cloud Run, 4 runs, min-instances=0, 5 to 10 minutes idle between each:

Run	Time
1	2.83s
2	13.62s
3	20.32s
4	3.23s

Figure 6: Real cold-start timings, unedited. The variance is part of the finding.

That spread is a real finding, and I'm not going to smooth it into an average and move on. 2.83s to 20.32s across cold starts that should be identical is wide. The idle gaps between runs weren't perfectly consistent, and I didn't isolate Vertex AI's own API-side latency from container cold-start time for these specific runs, so I can't attribute the spread precisely. Some of it might be Gemini Flash's own latency at Google's end, not Cloud Run's container startup at all.

GKE Autopilot, pod scale 0 to 1, timed via kubectl wait, then a timed request once ready:

Run	Pod scale-up	Request
1	3.020s	0.024s
2	2.882s	0.060s
3	3.703s	0.041s
4	3.450s	0.025s

Figure 7: Scale-from-zero timed across four runs, verified against a real Vertex AI response rather than trusted on timing alone.

Worth explaining why there's no dramatic first run outlier here. Early in this build, a Workload Identity permissions gap meant every request to the GKE deployment was failing with a 403, including my first cold-start pass, which returned fast because it was failing fast, not completing a real call. Once the identity issue was fixed and I reran the test against a confirmed working endpoint, the numbers below are what I got: no fresh-node spike, likely because the node was already warm from earlier testing by that point, not because the effect doesn't exist.

GKE doesn't scale to zero natively. Every number in the GKE table above comes from me manually scaling a Deployment between 0 and 1 replicas with kubectl scale, standing in for the zero-infrastructure-cost idle state that Cloud Run's min-instances=0 gives you as a first-class primitive. Autopilot still bills for cluster management regardless, and a real production deployment would more likely run a small non-zero minimum than actually scale pods to zero.²

A real Gemini Flash call through the GKE deployment, checked separately, came back with model_call_seconds: 2.278 and the response "Hello, nice to meet you." Confirmation that this dataset reflects genuine successful calls through a working identity chain, not fast failures dressed up as timings.

Load Test

One traffic pattern, not the three I'd originally planned, and I want to name that as a deliberate call rather than let it slide by as an unmentioned scope cut. Steady-moderate load, roughly 240 requests over about two minutes, run separately against each platform rather than at the same time. This isn't a head-to-head resource-contention test, just each platform's own behavior under the same shape of traffic.

Figure 8: Same load shape, two different metrics: Cloud Run's own request-rate curve against GKE's container CPU-time curve.

Cloud Run's request-count metric ramps cleanly to roughly 2 requests/second, peaks sharply, then drops off as the burst ends. GKE's CPU usage time metric tells the same story structurally: ramp, a sustained plateau through the load window, then the same drop-off, just measured as CPU-time rather than a request rate, since that's what the console actually gave me for this metric. I'm reading the shape off the chart here rather than quoting an exact peak figure, since I don't have a precise value pulled from the underlying data, only the rendered graph.

Cost: I'm deliberately not quoting a session total here. A short, low-volume test session doesn't represent what either platform actually costs to run at real traffic. If cost is the deciding factor for your workload, model it against your own traffic pattern and Vertex AI call volume in a production-like environment rather than against anything in this article.

The gVisor Example

This is where GKE Autopilot's extra complexity stops being overhead and turns into something you'd actually reach for.

# deployment.yaml (relevant excerpt)
spec:
  template:
    spec:
      runtimeClassName: gvisor
      serviceAccountName: adk-agent-ksa

kubectl apply -f deployment.yaml

kubectl get pod -l app=adk-agent -o jsonpath='{.items[0].spec.runtimeClassName}'

Figure 9: Confirmed from the live pod spec, not the manifest. The pod is actually running under gVisor.

That's the pod's live spec talking, not the YAML I wrote, proof it's actually running under gVisor's sandboxed runtime, not just asking nicely for one. GKE Autopilot lets you attach a RuntimeClass and get syscall-level sandbox isolation on a per-workload basis, chosen deliberately rather than inherited by default. Cloud Run isn't unsandboxed by any stretch (its own execution environment already runs on gVisor), but it doesn't give you this specific lever: the ability to pick and attach it, per workload, on purpose.³

For an ADK agent, that lever matters most in one specific case: a workload executing generated code, handling genuinely untrusted input, or otherwise needing a harder isolation boundary than the platform default gives you automatically. That's the real, specific reason to reach for GKE Autopilot here, not some vague sense that more control is always better.

My Actual Recommendation

Default to Cloud Run. The evidence from this build backs it up plainly: cold starts were broadly comparable between the two platforms once the corrected GKE data reflected real calls rather than fast failures, the identity story needed zero extra configuration on Cloud Run, and scale-to-zero is a real primitive rather than a kubectl scale script standing in for one. For the request-response shape most ADK agents actually have, GKE Autopilot's extra machinery just sits there unused.

Move to GKE Autopilot specifically when you need gVisor-level sandboxing or dedicated GPU node pools: node-level controls Cloud Run genuinely doesn't expose. Go in knowing about the Workload Identity step ahead of time: it's not optional, it produces a 403 with zero obvious connection to its actual cause if you haven't been warned. That's the honest price of the extra control, weighed against what that control actually buys you.

Limitations

Cloud Run's cold-start variance isn't fully explained, and I'd rather say so than round it away. 2.83s to 20.32s across runs that should be identical is a real spread, but I didn't isolate container startup time from Vertex AI's own API-side latency for these specific runs. A cleaner follow-up would instrument model_call_seconds on the Cloud Run side the way I did for one GKE call, to actually know how much of that spread is Cloud Run and how much is Gemini Flash itself.

GKE's scale-to-zero is simulated, not native. Every GKE cold-start number came from manually scaling a Deployment with kubectl scale, not a platform-level scale-to-zero primitive. Autopilot doesn't have one the way Cloud Run does.

Only one traffic pattern got load-tested: steady-moderate, roughly 240 requests over two minutes, run separately against each platform rather than concurrently. The original plan covered three patterns (bursty/idle, steady-low, steady-moderate); this is a deliberate scope cut, not something I'm pretending didn't happen.

Cost wasn't rigorously measured, and I'm not pretending otherwise. Validate cost against your own production traffic pattern and Vertex AI call volume before treating either platform as the cheaper option. A short test session isn't a substitute for that.

A short test session isn't sustained production load. A handful of cold-start runs and one two-minute load test give an honest snapshot of this specific workload's behavior on the day I ran it, not evidence of how either platform holds up under real, variable production traffic over days or weeks.

Key Takeaways

Cloud Run is the right default for most ADK agents. Faster cold starts in the common case, zero identity configuration, and genuine scale-to-zero rather than something you have to fake.
GKE Autopilot enforces Workload Identity by default, and it caught me out. A pod doesn't inherit the node's service account permissions the way Cloud Run's runtime does. Budget time for this if you choose GKE.
The actual reason to choose GKE Autopilot here is gVisor-level sandboxing or dedicated GPU node pools, not a general sense that more control is better. Pick it for a specific, named requirement Cloud Run doesn't expose.

Google Cloud. Use Workload Identity Federation for GKE. https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity ↩
Google Cloud. GKE pricing. https://cloud.google.com/kubernetes-engine/pricing ↩
Google Cloud. About GKE Sandbox (gVisor). https://cloud.google.com/kubernetes-engine/docs/concepts/sandbox-pods ↩

ADK Agent Identity on Google Cloud: Separate Identities, Separate Blast Radius

Dan Henderson — Wed, 15 Jul 2026 23:00:00 +0000

When a model-driven orchestrator delegates a task to a specialist agent, something specific happens at the infrastructure layer: one Cloud Run service makes an authenticated HTTP call to another. The caller presents its own service account token. The callee checks IAM before accepting anything.

What doesn't happen matters just as much. The orchestrator's identity isn't inherited by the sub-agent. The sub-agent runs as itself (its own service account, its own IAM scope), and whatever the orchestrator is permitted to do has no bearing on what the sub-agent is permitted to do. That's the architecture. Most multi-agent implementations don't make it explicit, and a fair number quietly assume the opposite.

In the Antigravity article that separation was a conceptual argument about governance. Here it's two Cloud Run services, two service accounts, and a specific IAM binding you can point at. And, as it turned out, a wrong default I had to catch and fix along the way.

Figure 1: Two Cloud Run services, two service accounts. The run.invoker binding is scoped to one named service, not the project, and that's what defines the blast radius in either direction.

Building the System in Antigravity

Two agents, both deliberately simple. The orchestrator receives a query and decides whether to answer it directly or delegate to a specialist. The specialist retrieves a document summary. Neither does anything more sophisticated than that, because the subject of this article is the identity layer sitting underneath them, not the reasoning inside them.

Antigravity scaffolds and deploys the sub-agent first, through agents-cli:

Use agents-cli to build a simple document retrieval specialist agent
using ADK. The agent accepts a query and returns a document summary.
Deploy to Cloud Run in europe-west2 with authentication required.
Name the service: doc-retrieval-agent

agents-cli create doc-retrieval-agent --prototype --yes
cd doc-retrieval-agent && agents-cli install
agents-cli scaffold enhance --deployment-target cloud_run --yes
agents-cli deploy

Figure 2: gemini-2.0-flash wasn't available in europe-west2, so agents-cli fell back to gemini-2.5-flash on its own, a regional constraint I'd otherwise have had to catch by hand. Both agents run 2.5-flash from here on.

What agents-cli Gives You By Default

Both services deploy cleanly, both requiring authentication:

Figure 3: Two services, both authenticated at the ingress level. Nothing wrong with this picture yet.

But look at what identity they're actually running as:

gcloud run services describe doc-retrieval-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID \
  --format="value(spec.template.spec.serviceAccountName)"

gcloud run services describe orchestrator-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID \
  --format="value(spec.template.spec.serviceAccountName)"

Figure 4: Same answer, twice. Both services are running as the default Compute Engine service account.

That's the bit that made me stop and go back through the deploy. The default Compute Engine service account carries roles/editor at the project level, so two agents sharing it don't just share access: a compromised sub-agent has Editor rights across the entire project, identical to the orchestrator's. There's no blast radius here at all, because nothing separates the two agents to begin with.agents-cli gets you to a working deployment fast. It doesn't get you to a secure one automatically. The IAM design is left to you, and if you don't go looking for it, this is what ships.

The Fix: Separate Identities

Dedicated service accounts, one per agent, each holding only what that agent needs:

gcloud iam service-accounts create doc-retrieval-sa \
  --project=YOUR_PROJECT_ID --display-name="Doc Retrieval Sub-Agent"

gcloud iam service-accounts create orchestrator-sa \
  --project=YOUR_PROJECT_ID --display-name="Orchestrator Agent"

# Sub-agent: Vertex AI only
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:doc-retrieval-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

# Orchestrator: Vertex AI, plus invoker on the sub-agent service only
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:orchestrator-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

gcloud run services add-iam-policy-binding doc-retrieval-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID \
  --member="serviceAccount:orchestrator-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/run.invoker"

# Point each Cloud Run service at its dedicated SA
gcloud run services update doc-retrieval-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID \
  --service-account=doc-retrieval-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com

gcloud run services update orchestrator-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID \
  --service-account=orchestrator-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com

echo "doc-retrieval-agent SA:" && gcloud run services describe doc-retrieval-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID --format="value(spec.template.spec.serviceAccountName)"
echo "orchestrator-agent SA:" && gcloud run services describe orchestrator-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID --format="value(spec.template.spec.serviceAccountName)"

Figure 5: Different answers now. That's the whole fix, and it's the difference between Figure 4 and this one that the rest of the article's argument rests on.

The scoping decision inside that fix is the one that actually matters, at least to me: roles/run.invoker is granted on the sub-agent's specific service, not at the project level. Project-level would let the orchestrator invoke any Cloud Run service in YOUR_PROJECT_ID; service-level lets it invoke exactly one. The intent was always the second (which, to be clear, was the only intent I had), and getting there costs one extra flag.

Antigravity's own deploy flow confirms the same binding, from its own summary:

Figure 6: Antigravity gets the run.invoker binding right for the delegation, while still defaulting the service account underneath it to the project-wide Compute Engine one. It gets the binding right and still leaves the identity choice to you.

Testing the Delegation Chain

The ADK API on Cloud Run doesn't auto-create sessions. Each caller creates one explicitly before it can send a message:

export TOKEN=$(gcloud auth print-identity-token)

curl -s -X POST "$ORCH_URL/apps/app/users/user_001/sessions/session_010" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'

curl -s -X POST "$ORCH_URL/run_sse" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"app_name":"app","user_id":"user_001","session_id":"session_010","new_message":{"role":"user","parts":[{"text":"Retrieve a document summary on IAM best practices for Cloud Run"}]},"streaming":false}'

Figure 7: The delegation shows up in the response itself: transfer_to_agent fires with agent_name: doc_retrieval_agent, and the synthesised summary comes back through the orchestrator.

Worth flagging: app_name in these calls is "app", not orchestrator_agent, which is what agents-cli names it by default on Cloud Run, regardless of what the agent itself is called.

A general question, run the same way, shows the orchestrator isn't delegating everything by reflex:

curl -s -X POST "$ORCH_URL/run_sse" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"app_name":"app","user_id":"user_001","session_id":"session_008","new_message":{"role":"user","parts":[{"text":"What is the capital of France?"}]},"streaming":false}'

Figure 8: No delegation, no A2A call, nothing sent to the sub-agent. The routing decision is real, not a formality.

Observability: What the Trace Shows

Cloud Trace picks up the A2A boundary crossing when the orchestrator initialises its connection to the sub-agent:

Figure 9: The agent-card discovery call RemoteA2aAgent makes on initialisation: GET /a2a/app/.well-known/agent-card.json, 200, 9.561ms.

I want to be honest about what this does and doesn't show, rather than let the screenshot imply more than it does. It confirms the delegation happened and that the orchestrator authenticated successfully to the sub-agent, which is the useful part. It doesn't show the sub-agent's own LLM call as a separate span here: Cloud Trace is capturing the HTTP boundary crossing, not the work happening once the request lands. The path itself is also worth a note: /a2a/app/.well-known/agent-card.json, not the /.well-known/agent.json the A2A spec describes elsewhere. Took me a minute to find it the first time. agents-cli's Cloud Run scaffold nests it under the app route.

The Blast Radius Proof

Everything above is a claim about what the IAM bindings allow. Two things prove it.

First, an unauthenticated call against the orchestrator:

curl -s -X POST "$ORCH_URL/run_sse" \
  -H "Content-Type: application/json" \
  -d '{"app_name":"app","user_id":"u","session_id":"s","new_message":{"role":"user","parts":[{"text":"test"}]}}' \
  -w "\nHTTP Status: %{http_code}\n"

Figure 10: 403. The authentication requirement is being enforced, not just configured and hoped for.

Second, and this is the one that actually answers the question the whole article is asking, who can invoke the orchestrator:

gcloud run services get-iam-policy orchestrator-agent \
  --region=europe-west2 --project=YOUR_PROJECT_ID \
  --format="table(bindings.members, bindings.role)"

Figure 11: Nothing. No member holds run.invoker on the orchestrator, not the sub-agent's service account, not anything else.

That empty table is the blast radius proof. The sub-agent has no IAM path back to the orchestrator at all. Even if it were compromised and tried to escalate by calling the orchestrator directly, there's no binding that would let the request through. The boundary is sitting in IAM, not in how well-behaved the agent's code happens to be.

Limitations

RemoteA2aAgent is experimental. ADK logs a UserWarning on startup that its A2A support (RemoteA2aAgent, A2aAgentExecutor, and the surrounding pieces) is experimental and subject to breaking changes. Not a reason to avoid it, in my view. Just a reason to pin your ADK version and re-test after every upgrade.

Agent card discovery needed public access, and this is the one that bugged me most. For the orchestrator to fetch /a2a/app/.well-known/agent-card.json at initialisation, that endpoint had to be reachable without an identity token, meaning allUsers gets run.invoker on the sub-agent, loosening exactly the control this article spends most of its length arguing for. In production, the better path is a private VPC with internal-only A2A traffic, or an authenticated agent-card fetch once ADK supports one cleanly.

Fixing the default service account once doesn't make it disappear. agents-cli deploys onto the default Compute Engine service account, roles/editor and all, unless you explicitly point it elsewhere. Creating dedicated accounts for these two agents doesn't strip that default account's standing access. Anything else in the project still using it keeps the wide blast radius alive.

Identity isn't the same as intent. Dedicated service accounts define what each agent can do. They don't define what each agent should do. A sub-agent with perfectly scoped IAM can still be prompted into acting outside its intended purpose. That's a policy problem sitting above IAM, and it's what the Agent Registry and Agent Gateway are for, not this layer.

Sessions are explicit, not automatic. The ADK API on Cloud Run requires a session to be created via POST /apps/{app_name}/users/{user_id}/sessions/{session_id} before /run_sse will accept a message. Not a limitation so much as a deployment detail worth knowing before your first request mysteriously fails.

Closing

Two agents with separate identities isn't a complex architecture, and I don't think it needs to be sold as one. It's just the simplest correct setup for a multi-agent system that takes governance seriously. And it's not the default you get for free. agents-cli handed both agents the same project-wide identity until I went and separated them myself, and I'd rather flag that gap between "deployed" and "correctly scoped" than pretend I caught it on the first pass.

The orchestrator can call the sub-agent because IAM says so. The sub-agent can't call the orchestrator because IAM says so: the empty policy table proves it, not my say-so. Neither can exceed its own scope, because IAM says so, not because the code is well-behaved and not because nothing has gone wrong yet.

That's not complete governance. But it's the right foundation, in the same spirit as the identity work in the BigQuery build: get the identity layer right first, and catch the defaults that don't. Everything else gets built on top of it.