DEV Community: Veer Muchandi

Under the Hood: Building Reliable Agentic Workflows

Veer Muchandi — Wed, 12 Aug 2026 16:25:17 +0000

When people first hear about AI agents, their initial reaction is almost always a mix of excitement and deep anxiety.

"How do we control it?"
"What if the AI hallucinates a critical step in a bank transaction?"
"Are we just letting an unpredictable black box run wild on our production databases?"

These are valid, vital questions. If you are building enterprise software, unpredictable is unacceptable. You cannot deploy systems that guess how to process payments, bypass compliance rules, or hallucinate shipping addresses.

But here is the truth: Successful agentic applications do not choose between absolute predictability and absolute flexibility. They combine both.

Let's pull back the curtain and look at the engineering under the hood of agentic orchestration—how we build safety, reliability, and human control directly into the architecture, and the critical infrastructure capabilities required to run, govern, and secure these dynamic systems at scale.

The Hybrid Model: Brains and Blocks

A common misconception is that agentic applications are completely non-deterministic. In reality, they are hybrid systems:

The LLM is the "Brain": It manages the reasoning, understands the unstructured input, plans the sequence, and decides which tool to invoke next. It is fluid and adaptable.
Microservices are the "Predictable Blocks": The tools, databases, and APIs that the LLM invokes remain completely deterministic. The agent cannot rewrite the code inside your payment API or change the security policies of your database.

By treating your existing APIs as semantic tools, the LLM acts as the intelligent tissue connecting highly reliable, predictable building blocks. If the brain decides to run a database query, the query itself is executed by a deterministic gateway that respects all your standard enterprise access controls and schema limits.

Comparing the Eras: Deterministic vs. Non-Deterministic Workflows

At an architectural level, the core functionality of a legacy application is implemented using hardcoded microservices with fully deterministic logic. Decisions about when to call a specific API are strictly predetermined by developers. The user interface in these systems is typically system-driven, offering rigid menus that constrain user interaction to predefined flows.

Agentic applications, on the other hand, build their core capabilities using AI Agents driven by LLM reasoning, thinking, and generative properties. Instead of hardcoding API call sequences, APIs are registered as semantic tools that the LLM invokes dynamically as needed. This shift enables a conversational or hybrid frontend that can adapt in real-time to follow the user's unstructured intent.

To summarize this shift, let's compare the fundamental dimensions of both approaches:

Dimension	Deterministic (Legacy)	Non-Deterministic (Agentic)
System Boundary	Fixed; developer-mapped paths only.	Fluid; dynamically solves unexpected permutations.
Maintenance	Increases exponentially with every edge case.	Flat; the LLM manages and discovers tools dynamically.
Input Format	Rigid schemas (JSON/gRPC).	Natural language, documents, and unstructured files.
Best For	Core ledger transactions, compliance, fixed data pipelines.	Contextual analysis, customer support, dynamic actions.

The Spectrum of Determinism

In practice, you rarely deploy pure, unrestrained non-determinism. Depending on the risk and complexity of the task, you select where your workflow sits on the Spectrum of Determinism:

Highly Deterministic (Legacy Workflows): Every path is hardcoded (e.g., BPMN engines). There is zero flexibility.
Semi-Deterministic (Orchestrated Agent Workflows): You use code-based frameworks like the Agent Development Kit (ADK) to establish guardrails. For example, you can enforce that Agent A must always complete its task before Agent B is invoked, but you allow each agent to use reasoning to solve its individual sub-task.
Non-Deterministic (LLM-Driven Orchestration): The LLM is given a catalog of tools (using open standards like the Model Context Protocol - MCP) and a goal. It dynamically plans and executes the entire path on the fly.

By using frameworks like the ADK, you can establish strict rules (like sequential or parallel gates) where needed, while still leaving space for the LLM to handle translation and unstructured reasoning.

NOTE
When discussing non-determinism in this spectrum, we are referring specifically to the flow and orchestration logic that decides how tasks are executed and which microservices are called. The underlying microservices, database transactions, and API endpoints themselves remain fully deterministic and predictable.

Human-in-the-Loop (HITL) Integration

No matter how advanced the AI, critical business processes will always require human eyes. Agentic applications integrate human review using two distinct architectural patterns:

1. Synchronous Intervention (Gating)

When an agent encounters a step that requires permission—such as approving a refund over $500 or signing off on a new contract—the workflow pauses and sleeps. It registers the task state in a persistent database, notifies the supervisor (via email, Slack, or a web portal), and waits. Once the human clicks "Approve," the workflow wakes up, loads the state, and continues execution.

2. Asynchronous Intervention (Exception Handling)

For background workflows running at scale (like processing 10,000 invoices overnight), pausing for every minor discrepancy is inefficient. Instead, the agent writes exceptions to a dedicated audit bucket and moves on to the next invoice. Humans review the exception bucket in the morning, correcting issues without halting the entire system.

Bringing It All Together: The Complete Agentic Ecosystem

Now that we have examined the individual gears—the LLM brain, the deterministic microservice blocks, the workflow spectrum, and the human gates—we can step back and see how they form a single, cohesive ecosystem.

Based on this architecture, we can define the full scope of a production-ready Agentic Application:

An agentic application is the combination of these different elements acting together within a single system boundary to solve business goals:

Conversational Agents managing active, real-time dialogues with users.
Background Agents executing silent, asynchronous system tasks.
Workflows orchestrating conditions, sequential/parallel paths, and loops between agents.
Human-in-the-Loop (HITL) Gates securing steps that require approval or exception auditing.

To keep this entire cooperative ecosystem running smoothly, reliably, and securely, we need to look beneath the workflows at the underlying infrastructure layer that powers it.

The Enterprise Plumbing: Required Infrastructure Capabilities

When building traditional enterprise applications, your infrastructure needs are relatively straightforward: web servers, compute instances, and database connections.

Agentic applications require all of these, but they also demand a brand new layer of infrastructure. Because agents reason dynamically, discover tools, and run asynchronous processes, they need a dedicated platform stack to run safely at scale:

1. The Core Agent Layer

AI Models: High-performance foundation models (LLMs and multimodal models) that provide the core reasoning, planning, and language generation engines.
Tool Registry: A secure directory housing all available API schemas, Model Context Protocol (MCP) servers, database connector specs, and custom execution functions that the agents can dynamically lookup and invoke.

2. Underlying Services Layer

Data & Knowledge Persistence: Persistent databases and file storage (like Cloud SQL or Google Cloud Storage) to store agent logs, session metadata, configurations, and grounding files.
Asynchronous Messaging (Pub/Sub): Event-driven brokers to coordinate background agents, triggering tasks when system events occur (like file uploads).
Scheduler: A timer engine to orchestrate recurring or timed agent executions.

3. Platform & Security Layer

Agent Identity: A unique security principal (similar to an enterprise service account) assigned to each agent. This allows the agent to authenticate itself to other services, establish clear audit trails, and declare its own security boundary.
Access Control (IAM): Governance frameworks that enforce security policies, ensuring agents only invoke tools and access databases that both the agent and the active user are explicitly authorized to use.
Managed Runtime & Sandboxes: A secure, isolated compute environment (like sandboxed containers) that executes agent code and prevents host server access. The runtime is intrinsically responsible for managing:
- Session Management: Tracking active user connection threads, managing connection lifecycles, and securing user authentication tokens.
- State & Memory Management: Maintaining the agent's internal state variables, execution states, planning history, and retrieval context (both short-term context windows and long-term memory banks).
Network Connectivity & Policy Control: Secure gateways, firewalls, and proxies that govern agent outgoing traffic, allowing administrators to apply network-level policies and configure exactly which external domains, APIs, and model endpoints the agents are allowed to access.
Access Portals: Unified web and mobile interfaces that allow users to discover and chat with conversational agents.
Observability & Telemetry: Audit trails and real-time logs recording the LLM's raw reasoning paths, prompt histories, tool calls, and API responses so developers can debug or audit system decisions.

Simplifying the Plumbing: The Gemini Enterprise Agent Platform (GEAP)

Rather than forcing engineering teams to construct, secure, and integrate these complex infrastructure layers from scratch, the Gemini Enterprise Agent Platform (GEAP) provides a unified, end-to-end runtime environment designed specifically for agentic applications.

GEAP structures these services into four core functional pillars that manage the entire agent lifecycle:

1. Build: Connecting Intelligence and Tools

At the construction layer, GEAP provides the frameworks and asset catalogs to connect your agent's brain to the physical world:

Gemini API & Model Garden: Access to first-party (Gemini), third-party, and open-source foundation models for inference and custom fine-tuning.
Tools & Datastore Connectors: Support for open protocols like the Model Context Protocol (MCP), Google Search grounding, Retrieval-Augmented Generation (RAG) datastores, and direct API endpoints.
Agent Development Kit (ADK): The framework code that enables developers to define agent intents, handle states, and run workflows programmatically.

2. Scale: Compute and Persistence

To run agents in production, GEAP coordinates the underlying Google Cloud infrastructure dynamically:

Agent Runtime & Sandboxes: Managed, isolated runtime sandboxes that spin up on demand to execute agent code securely without risking host infrastructure.
Asynchronous Mechanisms: Event-driven architectures powered by Google Cloud Pub/Sub and Eventarc to trigger background agents instantly when events occur.
State & Memory Management: Native session context storage combined with scalable Google Cloud storage databases—including Cloud SQL, AlloyDB, and Google Cloud Storage (GCS)—to handle long-term agent memory.

3. Govern: Enterprise-Grade Control

GEAP enforces strict security boundaries around agent execution to protect customer trust and proprietary data:

Agent Gateway & Registry: A centralized tool registry and gateway to register, manage, and audit all corporate agents.
Agent Identity & IAM: Unique security identities for every agent, ensuring they respect enterprise permissions.
Model Armor & Policies: Security filters that protect against prompt injection, enforce compliance rules, and detect anomalies.

4. Optimize: Continuous Improvement

Agents are evaluated and tuned continuously to maintain quality:

Agent Observability: Real-time dashboards monitoring the exact reasoning pathways, inputs, and outputs of active agents.
Evaluation & Simulation: Running synthetic user simulations and evaluations to verify changes to prompt configurations and tool behaviors before going to production.

Distributing and Consuming Agents: Portals and Channels

An enterprise agent is only valuable if users can easily access and interact with it. Gemini Enterprise provides multiple pathways to publish and consume agents depending on who the audience is:

The Gemini Enterprise App (The Employee Portal): For internal enterprise users, the Gemini Enterprise App serves as the central hub.
- Agent Gallery: A curated directory where employees can discover, access, and launch authorized agents that have been built and shared across the organization.
- Agent Designer: An intuitive interface within the platform that allows users to build no-code agents, compile their configurations, and publish them directly to the gallery. It also provides the capability to combine multiple agents and design multi-agent workflows.
Custom Web Applications (The External Portal): For external audiences like customers or partners, enterprises can build and host custom web applications (which can be hosted on any of the Google Cloud compute runtimes discussed below, such as Cloud Run or GKE). These custom apps consume the agents programmatically via APIs, embedding the agent's conversational capabilities directly into public-facing websites, customer portals, or mobile apps without requiring users to have enterprise credentials.

GEAP and the Broader Google Cloud Ecosystem

While GEAP provides the specialized agent runtimes, tool registries, and governance layers, a complete agentic application relies on the broader Google Cloud Ecosystem to persist data, manage asynchronous events, and schedule actions.

Here is how GEAP orchestrates these core Google Cloud services to build a production-ready application stack:

Data Persistence & Knowledge Bases: Relational databases (like Cloud SQL and AlloyDB) store transaction records, session metadata, and RAG vector indices, while object storage (like Google Cloud Storage (GCS)) persists raw unstructured documents.
Asynchronous Event Routing: High-throughput messaging queues (such as Google Cloud Pub/Sub) and event filters (such as Eventarc) trigger background agents automatically when system events occur.
Automation Schedulers & Business Workflows: Managed schedulers (like Cloud Scheduler) trigger background tasks on recurring timers, while orchestration engines (like Google Cloud Workflows) execute the deterministic business logic that agents invoke as tools.
Deterministic Compute Environments: While your deterministic microservices and APIs can run anywhere, Google Cloud offers highly scalable compute architectures to host them—including serverless runtimes (like Cloud Run), managed Kubernetes clusters (like Google Kubernetes Engine (GKE)), or standard virtual machine instances (like Google Compute Engine (GCE)).

By combining the specialized orchestration and governance capabilities of the Gemini Enterprise Agent Platform, the user-facing discovery and gallery portal of the Gemini Enterprise App, and Google Cloud's robust database, messaging, scheduling, and compute systems, Google Cloud provides a complete, integrated stack for agentic applications. This unified approach allows organizations to build, distribute, and execute flexible, self-healing systems without compromising on enterprise-grade security, scalability, or control.

📌 Key Takeaway

[!IMPORTANT]
"Enterprise-ready agentic orchestration is a team sport. It requires a hybrid model where fluid LLM reasoning is safely containerized inside deterministic microservices, managed runtimes, network policies, and human-in-the-loop gates. By combining the governance of the Gemini Enterprise Agent Platform with the proven strength of Google Cloud's data, messaging, and compute systems, you get a complete platform to build, scale, and secure the next generation of software."

In our third and final post, we will look at how you can start migrating your existing enterprise apps to this architecture today—gradually and safely, without a risky rip-and-replace.

Stay tuned for Part 3: Migrating from Microservices to Agentic Apps.

From AI Agents to Agentic Applications: Building Enterprise-Ready Systems

Veer Muchandi — Fri, 24 Jul 2026 21:22:14 +0000

If you are building or architecting enterprise software today, you are likely feeling a familiar, exhausting tension.

Your engineering teams are drowning in the complexity of hardcoding integration logic, handling messy unstructured data, and writing endless error-handling loops for microservices. We build pipelines, we write tests, we create fallback after fallback—and yet, the moment something unexpected happens, the system breaks.

To solve this, the industry is making a massive, collective shift toward AI Agents. Developers are racing to build agents that can reason, use tools, and interact dynamically.

But here is the hard truth: a standalone AI Agent is just an island.

A business process is rarely a single, isolated chat window. It spans multiple backend systems, background tasks, strict security policies, databases, and human approvals. If you just deploy a collection of disconnected agents, you haven't solved the enterprise problem; you have just created a new kind of "agent sprawl."

To make AI agents truly enterprise-ready, we must look beyond the agents themselves. The next frontier is about a fundamental shift in how applications are constructed.

Welcome to the era of the Agentic Application.

What is an Agentic Application?

At its core, an Agentic Application is a software system built by combining multiple specialized AI agents alongside structured orchestrational workflows to solve complex, non-deterministic business problems.

It represents a major paradigm shift in enterprise software development. By leveraging the reasoning capabilities of Large Language Models (LLMs) to stitch together deterministic microservices, agentic applications allow you to build fluid systems that adapt to unstructured human intent.

Instead of relying on a single, massive language model to do everything, an agentic application orchestrates a team of specialized agents. These agents generally fall into two categories:

Agent Type	Operational Mode	Trigger Mechanism	Enterprise Example
💬 Conversational Agents	Active & Chatty Interactive, real-time user-agent dialogue with fast response times.	Explicit user invocation (e.g., chat interfaces, voice commands).	A Sales Assistant Agent that helps a representative retrieve client history and draft emails.
⚙️ Background Agents	Asynchronous & Silent Executes complex, multi-step tasks independently and sleeps when finished.	System events, file uploads, timers, or schedulers.	A Sales Consolidator Agent that triggers when a regional sales file is uploaded, parsing and saving it.

Workflows and Orchestration: The Connective Tissue

AI agents rarely work in isolation. To solve complex business goals, they must be coordinated. Workflows serve as the orchestrational backbone of an agentic application, allowing multiple specialized agents to collaborate dynamically.

These agentic workflows support standard execution patterns like:

Sequential execution: One agent finishes its task and passes the output to the next.
Parallel execution: Multiple agents work on sub-tasks simultaneously to speed up processing.
Loops and Conditions: The system repeats or branches actions based on dynamic agent decisions or validations.

Depending on your architecture, orchestration generally falls into two patterns:

Code Orchestration: Written directly within the application codebase using agent frameworks like the Agent Development Kit (ADK) to manage flow logic programmatically.
External Orchestration: Managed by external orchestrators—such as visual workflow designs created in the Gemini Enterprise Agent Designer—to coordinate business processes across systems.

An agentic application is the intelligent fabric that weaves these agents and workflows together with underlying enterprise services—like persistent databases, secure identity access, and message queues—to solve real business problems.

A Real-World Example: The Sales Pipeline

To see how conversational agents, background agents, and workflows come together in production, let's look at a modern Sales Pipeline application:

In this scenario:

The Trigger: A regional sales file is uploaded to a storage bucket.
The Background Agent (Sales Consolidator): An asynchronous agent wakes up to parse the file and validate the formatting.
- The Happy Path: If the data is valid, it is ingested into Database, triggering updates in the Sales Visualizer dashboard.
- The Exception Loop: If the data is invalid, it is routed to the Sales Error Handler Agent, which isolates the issue and prompts a representative to correct and resubmit.

Here is a brief video walkthrough of this automated pipeline in action:

You can find the complete source code for this implementation in the sales_data_agentic_app GitHub repository.

Why Shift?

Let us understand the Limits of Traditional Deterministic Architectures

Traditional deterministic architectures (like microservices) are hitting operational ceilings. When a software application is required to handle unstructured human intent, unpredictable external API behaviors, and complex, multi-path user goals, legacy coding patterns begin to break down under their own weight.

Here are the three core architectural pain points of traditional systems, and how agentic design solves them:

1. Defeating the "Combinatorial Explosion" of Workflows

In a traditional microservices application, if you want to support multiple paths based on user input, you have to write rigid logic. You write if/else statements, switch blocks, or build complex state machines.

The Legacy Bottleneck: If you have 10 microservices that can be called in any order depending on a user's highly specific goal, the potential execution paths explode exponentially. You cannot realistically write, test, and maintain code for 10,000 different workflow variations.
The Agentic Leap: You write zero workflow paths. Instead, you register your 10 microservices as tools (using open standards like the Model Context Protocol - MCP) and give the LLM orchestrator a goal. The LLM handles the orchestration logic on the fly, dynamically creating a custom execution path tailored to that exact user, and discarding it when the goal is met.

2. Unstructured-to-Structured Translation

Human intent and real-world enterprise data are notoriously messy. Traditional systems require perfectly structured payloads (like strict JSON schemas or gRPC) to make routing decisions.

The Legacy Bottleneck: If a user says, "Look at my last invoice, find why the shipping charge is higher than our contract says, and draft a refund request," a traditional system chokes. You would need to build a specialized parsing service, an invoice lookup service, a contract comparison engine, and a routing matrix to tie them together.
The Agentic Leap: The LLM natively understands unstructured text and files. It extracts the variables, calls an invoice retrieval tool, reads the raw text of the contract within its context window, notices the pricing discrepancy, and dynamically chooses to call a draft refund tool—all without a single line of hardcoded integration logic.

3. Autonomous "Self-Healing" Loops

When a microservice fails in a deterministic application, your options are limited to what you coded ahead of time: retry three times, throw a 500 error, or trigger a complex compensating transaction.

The Legacy Bottleneck: If an API returns an unexpected validation error like 400: Invalid Date Format - Expected YYYY-MM-DD, a traditional orchestrator fails unless a developer explicitly wrote an exception handler for that specific endpoint.
The Agentic Leap: The LLM receives the error message back as a tool response. Because it understands language, it reads the error, realizes "Ah, I passed MM-DD-YYYY by mistake," autonomously corrects the payload format, and retries the tool call successfully. The system heals itself without ever crashing the user's session.

Deterministic vs. Agentic: Knowing When to Shift

This is not to say that traditional microservices are obsolete. Rather, the goal is to use the right tool for the job.

Deterministic (Legacy) Systems remain the gold standard for high-compliance, low-variance tasks like ledger transactions, payroll processing, and fixed data pipelines.
Non-Deterministic (Agentic) Systems are built for fluid environments that require reasoning, context-rich analysis, unstructured data translation, and dynamic discovery.

Dimension	Deterministic (Legacy)	Non-Deterministic (Agentic)
System Boundary	Fixed; developer-mapped paths only.	Fluid; dynamically solves unexpected permutations.
Maintenance	Increases with every edge case.	Flat; LLM manages new tools (MCP) dynamically.
Input Format	Rigid schemas (JSON/gRPC).	Natural language and unstructured files.
Best For	Core transactions, compliance, fixed data.	Contextual analysis, support, dynamic actions.

The ultimate architectural strategy is a hybrid one: Use an intelligent agentic orchestrator as the "brain," but have it invoke microservices that are themselves completely deterministic. You get the flexibility of AI with the predictable safety of traditional software.

📌 Key Takeaway

"Agentic applications do not replace microservices—they orchestrate them. By treating APIs as semantic tools, LLMs act as the intelligent tissue that binds static backend systems into responsive, self-healing enterprise applications."

This conceptual structure aligns beautifully with modern enterprise platforms like the Gemini Enterprise Agent Platform (GEAP), which provides the underlying runtime, identity, security, and tool registry services needed to scale these applications.

In our next post, we will pull back the curtain and look at the technical architecture under the hood: how agent runtimes, memory banks, and tool registries come together to make agentic orchestration safe, secure, and scalable.

Stay tuned for Part 2: Under the Hood of Agentic Orchestration.