When people first hear about AI agents, their initial reaction is almost always a mix of excitement and deep anxiety.
"How do we control it?"
"What if the AI hallucinates a critical step in a bank transaction?"
"Are we just letting an unpredictable black box run wild on our production databases?"
These are valid, vital questions. If you are building enterprise software, unpredictable is unacceptable. You cannot deploy systems that guess how to process payments, bypass compliance rules, or hallucinate shipping addresses.
But here is the truth: Successful agentic applications do not choose between absolute predictability and absolute flexibility. They combine both.
Let's pull back the curtain and look at the engineering under the hood of agentic orchestration—how we build safety, reliability, and human control directly into the architecture, and the critical infrastructure capabilities required to run, govern, and secure these dynamic systems at scale.
The Hybrid Model: Brains and Blocks
A common misconception is that agentic applications are completely non-deterministic. In reality, they are hybrid systems:
- The LLM is the "Brain": It manages the reasoning, understands the unstructured input, plans the sequence, and decides which tool to invoke next. It is fluid and adaptable.
- Microservices are the "Predictable Blocks": The tools, databases, and APIs that the LLM invokes remain completely deterministic. The agent cannot rewrite the code inside your payment API or change the security policies of your database.
By treating your existing APIs as semantic tools, the LLM acts as the intelligent tissue connecting highly reliable, predictable building blocks. If the brain decides to run a database query, the query itself is executed by a deterministic gateway that respects all your standard enterprise access controls and schema limits.
Comparing the Eras: Deterministic vs. Non-Deterministic Workflows
At an architectural level, the core functionality of a legacy application is implemented using hardcoded microservices with fully deterministic logic. Decisions about when to call a specific API are strictly predetermined by developers. The user interface in these systems is typically system-driven, offering rigid menus that constrain user interaction to predefined flows.
Agentic applications, on the other hand, build their core capabilities using AI Agents driven by LLM reasoning, thinking, and generative properties. Instead of hardcoding API call sequences, APIs are registered as semantic tools that the LLM invokes dynamically as needed. This shift enables a conversational or hybrid frontend that can adapt in real-time to follow the user's unstructured intent.
To summarize this shift, let's compare the fundamental dimensions of both approaches:
| Dimension | Deterministic (Legacy) | Non-Deterministic (Agentic) |
|---|---|---|
| System Boundary | Fixed; developer-mapped paths only. | Fluid; dynamically solves unexpected permutations. |
| Maintenance | Increases exponentially with every edge case. | Flat; the LLM manages and discovers tools dynamically. |
| Input Format | Rigid schemas (JSON/gRPC). | Natural language, documents, and unstructured files. |
| Best For | Core ledger transactions, compliance, fixed data pipelines. | Contextual analysis, customer support, dynamic actions. |
The Spectrum of Determinism
In practice, you rarely deploy pure, unrestrained non-determinism. Depending on the risk and complexity of the task, you select where your workflow sits on the Spectrum of Determinism:
- Highly Deterministic (Legacy Workflows): Every path is hardcoded (e.g., BPMN engines). There is zero flexibility.
- Semi-Deterministic (Orchestrated Agent Workflows): You use code-based frameworks like the Agent Development Kit (ADK) to establish guardrails. For example, you can enforce that Agent A must always complete its task before Agent B is invoked, but you allow each agent to use reasoning to solve its individual sub-task.
- Non-Deterministic (LLM-Driven Orchestration): The LLM is given a catalog of tools (using open standards like the Model Context Protocol - MCP) and a goal. It dynamically plans and executes the entire path on the fly.
By using frameworks like the ADK, you can establish strict rules (like sequential or parallel gates) where needed, while still leaving space for the LLM to handle translation and unstructured reasoning.
NOTE
When discussing non-determinism in this spectrum, we are referring specifically to the flow and orchestration logic that decides how tasks are executed and which microservices are called. The underlying microservices, database transactions, and API endpoints themselves remain fully deterministic and predictable.
Human-in-the-Loop (HITL) Integration
No matter how advanced the AI, critical business processes will always require human eyes. Agentic applications integrate human review using two distinct architectural patterns:
1. Synchronous Intervention (Gating)
When an agent encounters a step that requires permission—such as approving a refund over $500 or signing off on a new contract—the workflow pauses and sleeps. It registers the task state in a persistent database, notifies the supervisor (via email, Slack, or a web portal), and waits. Once the human clicks "Approve," the workflow wakes up, loads the state, and continues execution.
2. Asynchronous Intervention (Exception Handling)
For background workflows running at scale (like processing 10,000 invoices overnight), pausing for every minor discrepancy is inefficient. Instead, the agent writes exceptions to a dedicated audit bucket and moves on to the next invoice. Humans review the exception bucket in the morning, correcting issues without halting the entire system.
Bringing It All Together: The Complete Agentic Ecosystem
Now that we have examined the individual gears—the LLM brain, the deterministic microservice blocks, the workflow spectrum, and the human gates—we can step back and see how they form a single, cohesive ecosystem.
Based on this architecture, we can define the full scope of a production-ready Agentic Application:
An agentic application is the combination of these different elements acting together within a single system boundary to solve business goals:
- Conversational Agents managing active, real-time dialogues with users.
- Background Agents executing silent, asynchronous system tasks.
- Workflows orchestrating conditions, sequential/parallel paths, and loops between agents.
- Human-in-the-Loop (HITL) Gates securing steps that require approval or exception auditing.
To keep this entire cooperative ecosystem running smoothly, reliably, and securely, we need to look beneath the workflows at the underlying infrastructure layer that powers it.
The Enterprise Plumbing: Required Infrastructure Capabilities
When building traditional enterprise applications, your infrastructure needs are relatively straightforward: web servers, compute instances, and database connections.
Agentic applications require all of these, but they also demand a brand new layer of infrastructure. Because agents reason dynamically, discover tools, and run asynchronous processes, they need a dedicated platform stack to run safely at scale:
1. The Core Agent Layer
- AI Models: High-performance foundation models (LLMs and multimodal models) that provide the core reasoning, planning, and language generation engines.
- Tool Registry: A secure directory housing all available API schemas, Model Context Protocol (MCP) servers, database connector specs, and custom execution functions that the agents can dynamically lookup and invoke.
2. Underlying Services Layer
- Data & Knowledge Persistence: Persistent databases and file storage (like Cloud SQL or Google Cloud Storage) to store agent logs, session metadata, configurations, and grounding files.
- Asynchronous Messaging (Pub/Sub): Event-driven brokers to coordinate background agents, triggering tasks when system events occur (like file uploads).
- Scheduler: A timer engine to orchestrate recurring or timed agent executions.
3. Platform & Security Layer
- Agent Identity: A unique security principal (similar to an enterprise service account) assigned to each agent. This allows the agent to authenticate itself to other services, establish clear audit trails, and declare its own security boundary.
- Access Control (IAM): Governance frameworks that enforce security policies, ensuring agents only invoke tools and access databases that both the agent and the active user are explicitly authorized to use.
- Managed Runtime & Sandboxes: A secure, isolated compute environment (like sandboxed containers) that executes agent code and prevents host server access. The runtime is intrinsically responsible for managing:
- Session Management: Tracking active user connection threads, managing connection lifecycles, and securing user authentication tokens.
- State & Memory Management: Maintaining the agent's internal state variables, execution states, planning history, and retrieval context (both short-term context windows and long-term memory banks).
- Network Connectivity & Policy Control: Secure gateways, firewalls, and proxies that govern agent outgoing traffic, allowing administrators to apply network-level policies and configure exactly which external domains, APIs, and model endpoints the agents are allowed to access.
- Access Portals: Unified web and mobile interfaces that allow users to discover and chat with conversational agents.
- Observability & Telemetry: Audit trails and real-time logs recording the LLM's raw reasoning paths, prompt histories, tool calls, and API responses so developers can debug or audit system decisions.
Simplifying the Plumbing: The Gemini Enterprise Agent Platform (GEAP)
Rather than forcing engineering teams to construct, secure, and integrate these complex infrastructure layers from scratch, the Gemini Enterprise Agent Platform (GEAP) provides a unified, end-to-end runtime environment designed specifically for agentic applications.
GEAP structures these services into four core functional pillars that manage the entire agent lifecycle:
1. Build: Connecting Intelligence and Tools
At the construction layer, GEAP provides the frameworks and asset catalogs to connect your agent's brain to the physical world:
- Gemini API & Model Garden: Access to first-party (Gemini), third-party, and open-source foundation models for inference and custom fine-tuning.
- Tools & Datastore Connectors: Support for open protocols like the Model Context Protocol (MCP), Google Search grounding, Retrieval-Augmented Generation (RAG) datastores, and direct API endpoints.
- Agent Development Kit (ADK): The framework code that enables developers to define agent intents, handle states, and run workflows programmatically.
2. Scale: Compute and Persistence
To run agents in production, GEAP coordinates the underlying Google Cloud infrastructure dynamically:
- Agent Runtime & Sandboxes: Managed, isolated runtime sandboxes that spin up on demand to execute agent code securely without risking host infrastructure.
- Asynchronous Mechanisms: Event-driven architectures powered by Google Cloud Pub/Sub and Eventarc to trigger background agents instantly when events occur.
- State & Memory Management: Native session context storage combined with scalable Google Cloud storage databases—including Cloud SQL, AlloyDB, and Google Cloud Storage (GCS)—to handle long-term agent memory.
3. Govern: Enterprise-Grade Control
GEAP enforces strict security boundaries around agent execution to protect customer trust and proprietary data:
- Agent Gateway & Registry: A centralized tool registry and gateway to register, manage, and audit all corporate agents.
- Agent Identity & IAM: Unique security identities for every agent, ensuring they respect enterprise permissions.
- Model Armor & Policies: Security filters that protect against prompt injection, enforce compliance rules, and detect anomalies.
4. Optimize: Continuous Improvement
Agents are evaluated and tuned continuously to maintain quality:
- Agent Observability: Real-time dashboards monitoring the exact reasoning pathways, inputs, and outputs of active agents.
- Evaluation & Simulation: Running synthetic user simulations and evaluations to verify changes to prompt configurations and tool behaviors before going to production.
Distributing and Consuming Agents: Portals and Channels
An enterprise agent is only valuable if users can easily access and interact with it. Gemini Enterprise provides multiple pathways to publish and consume agents depending on who the audience is:
- The Gemini Enterprise App (The Employee Portal): For internal enterprise users, the Gemini Enterprise App serves as the central hub.
- Agent Gallery: A curated directory where employees can discover, access, and launch authorized agents that have been built and shared across the organization.
- Agent Designer: An intuitive interface within the platform that allows users to build no-code agents, compile their configurations, and publish them directly to the gallery. It also provides the capability to combine multiple agents and design multi-agent workflows.
- Custom Web Applications (The External Portal): For external audiences like customers or partners, enterprises can build and host custom web applications (which can be hosted on any of the Google Cloud compute runtimes discussed below, such as Cloud Run or GKE). These custom apps consume the agents programmatically via APIs, embedding the agent's conversational capabilities directly into public-facing websites, customer portals, or mobile apps without requiring users to have enterprise credentials.
GEAP and the Broader Google Cloud Ecosystem
While GEAP provides the specialized agent runtimes, tool registries, and governance layers, a complete agentic application relies on the broader Google Cloud Ecosystem to persist data, manage asynchronous events, and schedule actions.
Here is how GEAP orchestrates these core Google Cloud services to build a production-ready application stack:
- Data Persistence & Knowledge Bases: Relational databases (like Cloud SQL and AlloyDB) store transaction records, session metadata, and RAG vector indices, while object storage (like Google Cloud Storage (GCS)) persists raw unstructured documents.
- Asynchronous Event Routing: High-throughput messaging queues (such as Google Cloud Pub/Sub) and event filters (such as Eventarc) trigger background agents automatically when system events occur.
- Automation Schedulers & Business Workflows: Managed schedulers (like Cloud Scheduler) trigger background tasks on recurring timers, while orchestration engines (like Google Cloud Workflows) execute the deterministic business logic that agents invoke as tools.
- Deterministic Compute Environments: While your deterministic microservices and APIs can run anywhere, Google Cloud offers highly scalable compute architectures to host them—including serverless runtimes (like Cloud Run), managed Kubernetes clusters (like Google Kubernetes Engine (GKE)), or standard virtual machine instances (like Google Compute Engine (GCE)).
By combining the specialized orchestration and governance capabilities of the Gemini Enterprise Agent Platform, the user-facing discovery and gallery portal of the Gemini Enterprise App, and Google Cloud's robust database, messaging, scheduling, and compute systems, Google Cloud provides a complete, integrated stack for agentic applications. This unified approach allows organizations to build, distribute, and execute flexible, self-healing systems without compromising on enterprise-grade security, scalability, or control.
📌 Key Takeaway
[!IMPORTANT]
"Enterprise-ready agentic orchestration is a team sport. It requires a hybrid model where fluid LLM reasoning is safely containerized inside deterministic microservices, managed runtimes, network policies, and human-in-the-loop gates. By combining the governance of the Gemini Enterprise Agent Platform with the proven strength of Google Cloud's data, messaging, and compute systems, you get a complete platform to build, scale, and secure the next generation of software."
In our third and final post, we will look at how you can start migrating your existing enterprise apps to this architecture today—gradually and safely, without a risky rip-and-replace.
Stay tuned for Part 3: Migrating from Microservices to Agentic Apps.
Top comments (0)