Designing a Production-Ready Bank Agentic System on Google Cloud

Designing an enterprise-grade AI agentic application requires balancing core AI engineering with robust software engineering principles. When moving from a simple conversational proof-of-concept to a fully productionized banking assistant, engineers must address key architectural questions:

1. What are the core AI vs. software engineering trade-offs?
2. Where does a single-agent system fail?
3. Why transition to a multi-agent system with domain-specific sub-agents
4. How does the Model Context Protocol (MCP) decouple reasoning from API integration?
5. Should we use a third-party LLM or host our own model?
6. How do we enforce edge security, zero-trust authentication/authorization, PII masking, and full observability on Google Cloud Platform (GCP)?

Below is a step-by-step walkthrough detailing how to evolve a basic chatbot into an enterprise-ready, production-grade AI agentic system using native Google Cloud services.

Step 1: The Business Problem & Simple Chatbot Architecture
The Business Challenge

A major retail bank's customer support team reports receiving 400,000 calls per month, with an average call duration of 4 minutes. Detailed analysis reveals that 65% of these queries are routine:

1. Checking account balances.
2. Inquiring about recent account debits.
3. Requesting new cheque books.

While these responses are available within the bank's net banking app, complex navigation across 14+ screens drives customers to call toll-free support lines instead. Because the bank pays network providers for toll-free volume, this costs millions of dollars annually in telephone bills.

The goal is twofold: maintain a conversational customer experience while drastically reducing operational telephony costs by building an intelligent AI support bot. The Initial Simple Chatbot. We start with a baseline architecture: a front-end interface sending messages via a backend API to an LLM-backed agent.

Limitation
If a customer asks, "What is my account balance?", the agent responds with a fallback: "Sorry, I don't have access to your bank accounts or account balances." The LLM lacks integration with internal banking data.

Step 2: Enabling Tool Access & The Tool Overload Problem. To resolve customer inquiries, we connect the agent to internal banking APIs (e.g., balance inquiry, statement request, address change).

When a user asks, "What is my balance?", the agent forwards the user query alongside descriptions of all available tools to the LLM. The LLM selects the Balance Enquiry tool, the agent executes the backend REST/gRPC API call, retrieves the live balance, and the LLM formats a natural language response (e.g., "Your account balance is $12,400").

The Architectural Bottleneck: In an enterprise banking system, an agent frequently requires access to 20–35+ distinct APIs. Attaching every tool schema to a single agent causes tool confusion and tool overload, making it difficult for the LLM to reliably select the correct schema while drastically increasing token usage and latency.

Steps 3 & 4: Multi-Agent Architecture with Vertex AI Agent Engine To prevent tool overload, we decompose the system into specialized sub-agents coordinated by an Orchestration / Coordinator Agent:

1. Accounts Agent: Focuses exclusively on account balance and status inquiries.
2. Transaction Agent: Manages transaction history and statement generation.
3. Service Request Agent: Processes profile updates, KYC, and cheque book requests.

Google Cloud Implementation Details

1. Agent Framework & Runtime: Built using the code-first Agent Development Kit (ADK) and deployed as scalable container microservices on Cloud Run or fully managed via Vertex AI Agent Engine.
2. Orchestration Flow: For compound prompts like "What is my balance, and can you also send me my latest transactions?", the Coordinator Agent leverages Vertex AI Gemini 2.5 Pro to construct an execution plan:
3. Route to the Accounts Agent to retrieve balance details.
4. Route to the Transaction Agent to fetch transaction records.
5. Synthesize both outputs into a unified natural language answer.

Step 5: Decoupling Tools via Model Context Protocol (MCP) Directly hardcoding API client schemas, authentication header construction, parameter extraction, and error-handling logic into the agent codebase tightly couples API engineering with LLM reasoning.

We decouple these responsibilities by introducing Model Context Protocol (MCP) Servers running on Cloud Run. The agent focuses purely on reasoning, while MCP servers standardize tool execution, parameter parsing, and schema exposure.

Step 6: Authentication & Zero-Trust Authorization
In an unauthenticated system, the agent must prompt the user: "Please enter your Customer ID". If a user provides another individual's ID, the bot blindly returns that customer's private account details.

Google Cloud Implementation Details:

• Authentication: Users authenticate against the bank's Identity Platform (OAuth2/OIDC) before accessing the interface.
• Zero-Trust Identity Propagation: Once logged in, the user's authenticated JWT token passes through Apigee API Gateway to downstream agents. The system automatically extracts identity context; the bot never asks for raw customer IDs.
• Authorization: Fine-grained policy validation (GCP IAM Service Accounts & Policy Controller) verifies that the authenticated user explicitly owns the requested resources before executing any MCP tool.

Step 7: Memory & Session State Management
A stateless agent cannot cross-reference past interactions, preventing advanced capabilities like identifying fraudulent transaction patterns across multiple sessions.

Google Cloud Implementation Details

1. Short-Term & Inter-Agent State: Memorystore for Redis caches real-time session state and facilitates shared context across specialized agents. 2. Long-Term Memory Bank: Cloud Spanner or Cloud SQL persists long-term conversational records and interaction logs with multi-region ACID compliance, allowing agents to trace transaction disputes and historical inquiries.

Step 8: Data Privacy, PII Guardrails & Model Selection
To satisfy strict banking compliance regulations, sensitive Personal Identifiable Information (PII) must be sanitized before reaching third-party cloud models.

*Google Cloud Implementation Details: *

1. PII Masking: Prompts pass through Google Cloud Sensitive Data Protection (Cloud DLP) or Model Armor to redact 16-digit credit card numbers, Tax IDs, SSNs, and account numbers prior to LLM submission. Model Hosting Flexibility: i. Managed Endpoints: Vertex AI for Gemini 2.5 / Claude models. ii. Self-Hosted / Hybrid: Custom open-weights models (e.g., Gemma 2) deployed on Cloud Run or GKE with GPU acceleration for sensitive internal workloads.

Image Here in Step 8

Step 9: Edge Security, Guardrails & Reliability
Production banking platforms require robust defenses against prompt injection attacks, endpoint overload, and downstream service failures.

Google Cloud Implementation Details

1. Edge Layer Protection: Google Cloud Armor enforces WAF policies and mitigates DDoS attacks. Apigee manages strict rate limiting (e.g., max 4 requests/sec per user) to protect backend microservices.
2. Agent Guardrails & Validation: Input prompts are sanitized against prompt injection techniques. MCP servers validate input schemas before issuing requests to backend APIs.
3. Resilience Patterns: Retries, timeouts, and circuit breakers ensure gracefully handled service degradations.

Image Here Step 9:

Step 10: Full Enterprise Architecture & Observability

Observability in AI systems requires tracing prompt execution, agent routing decisions, tool call inputs/outputs, resource utilization, and operational costs. It is essential for Tracing, & Evaluation.
Cloud Trace & OpenTelemetry trace multi-agent execution steps and tool calls down to individual MCP HTTP requests. Additionally, it does Cloud Logging & Monitoring monitor system resource metrics (CPU, Memory, token usage).

Vertex AI Evaluation Service continuously audits agent response accuracy, grounding, and safety metrics.

Let me share the complete Google Cloud Bank's Production Architecture Diagram here