Introduction
As AI agents grow in capability, security, isolation, observability, and control need to be built into the underlying infrastructure for their operations. Multiple tenant workspaces running in a shared infrastructure are efficient from the perspective of cost efficiency, but pose significant challenges to the design. In this case, the problem lies in the ability to isolate tenants sufficiently while still enjoying the performance and high density that containers offer.
Traditional containers provide a way to run workloads that are relatively efficient but which are hosted on the same host kernel. If used internally in a system, it might work. In multi-tenancy scenarios, where the tenants are able to run tools, execute scripts or even have access to file systems and connect to the outside world, relying solely on containers becomes a vulnerability. It would make more sense to use a microVM that provides strong workload isolation while still offering efficiency. Firecracker is an implementation of microVMs based on VM technology with isolation guarantees and container performance characteristics.
The concept is easy to grasp: each tenant gets isolated resources (its own kernel, file system, disk volume, network boundaries). All tenancy processes including provisioning, scheduling, monitoring, backups, restores, authentication, and tenant management happen in the shared control plane, while the runtime remains isolated from the control plane itself.
Core Platform Concept
The architecture is defined by two planes.
The control plane handles identity, API, tenancy, scheduling, metadata, backup orchestration, auditing/logging, and administrative visibility.
The data plane runs the actual tenant environment. A tenant workspace runs in its own microVM with CPU, memory, disk, and networking resources assigned to it. A tenant’s runtime file system and kernel are separate from those of another tenant.
This type of approach is particularly useful for agent platforms where the agent runs arbitrary code, tools, files, calls out to APIs, browses the web, or accesses model providers. The stronger the agent, the bigger the risk posed by poor isolation. The design principle here shouldn’t be "How do we run lots of agents efficiently?" but rather "How do we run lots of agents without making one tenant a threat to other tenants?
Why MicroVMs Matter
A microVM gives each tenant a stronger isolation boundary than a container because each tenant gets a separate guest kernel and virtualized environment. This matters when tenants are allowed to execute commands, install packages, process documents, or use autonomous tooling.
The source pattern uses per-tenant microVM isolation, independent root filesystems, tenant-specific data volumes, network separation, scheduling, host scale-out, idle reclamation, health checks, web-based management, backup/restore, and shared skill/configuration distribution.
That is the right architecture direction. The bad version of this platform would be “one Kubernetes cluster, one namespace per tenant, and hope RBAC saves us.” That is not enough for serious agent workloads. Namespaces are management boundaries, not hard security boundaries.
Proposed GCP System Architecture
The GCP implementation should use a serverless control plane and a Compute Engine-based microVM data plane.
At the edge, users access the platform through HTTPS Load Balancing protected by Cloud Armor. Authentication is handled through Identity Platform. API Gateway exposes tenant lifecycle APIs, such as create tenant, stop tenant, delete tenant, query status, trigger backup, restore workspace, and list host capacity.
Cloud Run services implement the control logic. Firestore stores tenant metadata, host state, quota information, audit events, backup status, and lifecycle state. Cloud Storage stores root filesystem images, release bundles, tenant backup artifacts, configuration templates, and shared skill packages.
The tenant workloads run on a regional Managed Instance Group. Each host is a Compute Engine VM with nested virtualization enabled. During startup, the host downloads the approved runtime image, registers itself into the metadata store, reports capacity, and starts a host agent. When a tenant is created, the scheduler selects a healthy host with sufficient capacity and asks the host agent to create the tenant microVM.
Each tenant microVM receives its own filesystem, persistent data volume, network tap interface, CPU/memory allocation, and runtime configuration. The tenant dashboard is exposed through a controlled proxy path instead of giving every tenant a separate public endpoint.
Recommended GCP Reference Architecture
For production-grade direction, the recommended architecture is:
Identity Platform for authentication
Cloud Armor for WAF and DDoS protection
HTTPS Load Balancer for public entry
API Gateway for API routing and authorization
Cloud Run for control-plane services
Firestore for metadata and lifecycle state
Pub/Sub for asynchronous events
Cloud Scheduler for periodic automation
Cloud Storage for rootfs, templates, backups, and shared assets
Secret Manager for API keys and runtime secrets
Compute Engine regional Managed Instance Groups for microVM hosts
Nested virtualization enabled on host VMs
Cloud Monitoring and Logging for observability
Terraform for repeatable infrastructure deployment
This gives a clean separation between serverless management services and hardened tenant runtime infrastructure.
Main Risks
The biggest risk is pretending this is just another web application deployment. It is not. This is platform engineering.
The second risk is underestimating networking. Tenant microVM routing, private bridges, NAT, proxying dashboards, egress controls, and firewall rules must be designed carefully. Sloppy networking can destroy the isolation story.
The third risk is weak lifecycle management. Creating tenants is easy. Cleaning them up safely, preserving data, reclaiming idle resources, restoring backups, and handling failed starts is where the platform becomes real.
The fourth risk is image drift. If root filesystem images, host agents, and tenant configurations are not versioned, you will eventually have tenants behaving differently across hosts.
Conclusion
A secure multi-tenant AI agent platform on GCP is absolutely possible, but the right design is not a simple container platform. The stronger design uses a serverless control plane and a microVM-based data plane. GCP provides enough building blocks to implement this pattern, especially through Compute Engine nested virtualization, Managed Instance Groups, API Gateway, Cloud Run, Cloud Storage, Firestore, Pub/Sub, Identity Platform, Secret Manager, Cloud Armor, and Cloud Monitoring.
The blunt truth: if the target is a credible multi-tenant AI execution platform, do not cheap out on isolation. Containers alone are convenient, but convenience is not a security model. For AI agents that can execute tools and code, microVM-level isolation is the right architectural direction.
Top comments (0)