DEV Community: Varsha Das

The Java AI Stack Just Crystallized. Here's the Architecture That Emerged.

Varsha Das — Mon, 10 Aug 2026 10:26:56 +0000

If you're a Java developer reading this in August 2026, the landscape shifted under your feet in the last 90 days — and most of the internet hasn't caught up.

Spring AI hit 2.0 GA. Embabel reached 1.0. MCP went stateless. LangChain4j shipped BDI agents. The Spring AI AgentCore SDK went GA. And Koog (JetBrains) stabilized its core. All between May and July 2026. All for the JVM.

Catch up on — ai4jvm.com

This isn't a listicle of announcements. This is an architecture essay — the synthesis of a 68-minute studio conversation I had with James Ward. Java Champion. AWS Principal Developer Advocate. 12-Factor App contributor. Building software since 1997. We recorded 3 hours. Published 68 minutes. What made the cut was the stuff that changes how you think about building production systems — not demos.

🎬 Full 68-minute conversation

The Foundational Claim: Python Trains AI. Java Runs AI.

Let me ground this with numbers.

60% of external AWS SDK API calls come through the Java SDK. 8% come from Python. (Source: James Ward, "Going Big with Java for Agentic Workloads," 2026)
90% of Fortune 500 companies run Java in production. (Oracle, 2025)
62% of enterprises now use Java to power AI applications. (Azul survey, January 2026)
A Spring AI survey indicates 65% of Spring users prefer building agents in Spring AI/Embabel over Python/TypeScript alternatives (17%) or other JVM frameworks (18%).

The distinction matters because people conflate building with AI (integrating models into applications — the Java play) with building AI (training models — the Python play).

When an AI agent needs to call your payment gateway at 3am with proper retry logic, circuit breakers, type-safe validation, and an audit trail — that's a distributed systems problem. It always was. The model is a component in that system, not the system itself.

The agents of 2026 are calling the same enterprise APIs, hitting the same backends, running on the same infrastructure that Java has owned for two decades. The language that runs those backends is the language best positioned to serve them to autonomous consumers.

The Framework Decision: A Concrete Comparison (Not a Vibe Check)

The question I get most — at meetups, in YouTube comments, at the MCP Dev Summit booth (100+ conversations across Bangalore and Mumbai) — is: "Spring AI or LangChain4j?"

James's answer was blunt: pick one, go deep. The architectural concepts — tool calling, RAG, structured outputs, memory, observability — are transferable.

Here's the concrete state of play as of August 2026:

	Spring AI 2.0	LangChain4j 1.18	Embabel 1.0	Koog 1.0
GitHub Stars	7.4K	10K	2.9K	3.5K
GA Date	Jun 12, 2026	Jul 2026	Jul 20, 2026	Jul 2026
Foundation	Spring Boot 4	Standalone / Quarkus	Built ON Spring AI	Kotlin multiplatform
Agent Pattern	Advisor chain (composable interceptors)	BDI (Belief-Desire-Intention)	GOAP (Goal-Oriented Action Planning)	Type-safe DSL
MCP Support	Annotation-driven (@McpTool)	Declarative via @tool	Inherits from Spring AI	A2A + MCP
Key Differentiator	Enterprise convention-over-config	Broadest provider support (20+ LLMs, 20+ vector stores)	Runtime replanning when actions fail	Multiplatform (JVM, JS, iOS, Android)
Best For	Spring Boot teams (66% of enterprise Java)	Non-Spring shops, polyglot provider needs	Teams wanting declarative goal-driven agents	Kotlin-first teams

The framework matters less than understanding what's underneath. If you understand how tool calling works at the protocol level (MCP), how memory strategies differ (semantic vs episodic), and how retry responsibility splits between infra and reasoning — switching between these is a weekend of refactoring, not a rewrite.

The anti-pattern I see repeatedly: Teams spending 4 weeks evaluating frameworks, building proof-of-concepts in each, writing comparison docs for leadership — and never shipping anything. The framework you ship with is better than the three you evaluated.

MCP 2026–07–28: The Protocol Layer Just Broke and Rebuilt Itself

On July 28, 2026, the Model Context Protocol shipped its biggest revision since launch. This isn't a point release. It's a new era.

What changed (the technical diff):

The old protocol was stateful — a two-round-trip initialize handshake, a Mcp-Session-Id header that required sticky sessions, server-side session stores, and affinity routing at the load balancer.

The new protocol is stateless: every request is self-contained. Protocol version and client capabilities ride in _meta on every request. No handshake. No session ID.

// Before (2025-11-25): Two round trips minimum
POST /mcp → initialize → get Mcp-Session-Id → sticky-route forever

// After (2026-07-28): One round trip. Everything in the envelope.
POST /mcp
Mcp-Method: tools/call
Mcp-Name: get_user

{
  "jsonrpc":"2.0",
  "method":"tools/call",
  "params":{
    "name":"get_user",
    "arguments":{"id":"123"},
    "_meta":{
      "io.modelcontextprotocol/protocolVersion":"2026-07-28",
      "io.modelcontextprotocol/clientInfo":{"name":"my-client","version":"1.0.0"}
    }
  }
}

Why this matters for Java developers:

Your MCP server (the Spring Boot app with @McpTool annotations) can now sit behind a plain round-robin load balancer. No sticky sessions. No shared session store. No distributed cache for session state. The serverless deployment model (Lambda, AgentCore Runtime, Cloud Run) becomes trivially simple because there's nothing to share between invocations.

The MCP Java SDK 2.0 tracks this spec. SSE transport is deprecated (12-month clock). Streamable HTTP is the path forward. server/discover is now mandatory — it replaces the initialization handshake as the capability advertisement mechanism. Tool lists must be deterministic and carry ttlMs and cacheScope for client-side caching.

The migration reality: You have 12 months. Both eras coexist. But if you're building a NEW MCP server today, build to 2026–07–28. Don't accumulate session-dependent tech debt you'll need to unwind.

Embabel 1.0: Why Rod Johnson Built a Framework on Top of His Own Framework

The InfoQ coverage frames it precisely: "An analogy: Spring AI exists at the level of the Servlet API, while Embabel is more like Spring MVC."

Rod Johnson — who created Spring in 2003 — is making the same layering bet he made 23 years ago.

What makes Embabel architecturally different:

Embabel uses Goal-Oriented Action Planning (GOAP), a technique borrowed from video game AI. Instead of wiring a static graph of nodes and edges (the LangGraph approach), you define:

Typed goals — what you want to achieve
Typed actions — each with preconditions and effects
A planner — that searches for a sequence of actions reaching the goal at runtime

If the world changes mid-task (a tool fails, new data arrives), the planner reassesses and finds a new path. You don't need to have anticipated every branch in advance.

This matters for production because agents fail mid-execution constantly. An API returns a 503. The user changes their mind. A guardrail blocks an intermediate step. With a static graph, you either crash or you need explicit error edges for every possible failure. With GOAP, the planner routes around the failure — the same way a GPS recalculates when you miss a turn.

The distinction from LangGraph4j: LangGraph requires you to define the graph upfront — which nodes exist, which edges connect them, under what conditions. Embabel's planner discovers viable paths through typed actions at runtime. Both approaches have merit. LangGraph gives you predictability and auditability. Embabel gives you adaptability and less upfront wiring. Embabel even supports mixing both: GOAP planning alongside explicit state machines in the same agent.

Memory Is the Silent Killer of Agent Reliability

The most technically interesting segment of our conversation — and the one that's generated the most YouTube comments — was about agent memory architecture.

The failure mode nobody warns you about: Your agent keeps the last 20 messages in a sliding window. On message 21, the original user intent rolls off. The agent continues executing — but toward a goal it's forgotten. No crash. No error. Just quiet drift. Your trace shows every tool call succeeded. The output is confidently wrong.

The 4-strategy architecture:

Strategy	What It Stores	When It's Used	Example
Semantic	Factual information about the user	Injected when relevant to current query	"User works in financial services"
User Preference	Explicit settings and choices	Always injected for personalization	"Prefers metric units"
Summary	Condensed conversation history	Injected at session start for continuity	"Last session: analyzed Q3 expenses, found anomaly in travel category"
Episodic	Past interactions and lessons learned	Injected when similar situations arise	"User had trouble with CSV parsing last week — offer the helper tool proactively"

The production pattern: Short-term memory (sliding window) handles the current conversation. Long-term memory (the 4 strategies above) handles everything that should survive across sessions. The consolidation happens asynchronously — the system extracts relevant facts without explicit developer intervention.

The anti-pattern: Using only short-term memory and increasing the window size to compensate. This burns tokens (every message is re-sent to the model) and eventually hits context limits. It's the equivalent of solving a database scaling problem by buying more RAM instead of indexing properly.

The Principle I Now Repeat in Every Talk

"Don't let the non-deterministic system make the final move."

LLMs reason. Deterministic tools execute. The boundary between them must be type-safe, validated, and auditable.

In practice, this means your @McpTool methods (or @Tool in LangChain4j) have:

Strict input types (not Map<String, Object>)
Validation annotations (@NotNull, @Min, @Pattern)
Structured error responses the agent can reason about
No direct writes without validation passing first

@McpTool(description = "Transfer funds between accounts. Validates balance before executing.")
public TransferResult transfer(
    @NotNull @Pattern(regexp = "ACC-\\d{10}") String fromAccount,
    @NotNull @Pattern(regexp = "ACC-\\d{10}") String toAccount,
    @Min(1) @Max(100000) BigDecimal amount,
    @NotBlank String currency
) {
    // The agent reasoned about WHAT to transfer.
    // This method validates HOW — deterministically.
    // If validation fails, the structured error tells the agent why.
}

This is just dependency injection and interface contracts — the things Java developers have been doing for 20 years. The reasoning is new. The execution boundary isn't.

What This Means for Your Career in 2026

The Java AI stack crystallized in 90 days. Before May 2026, it was emerging. Now it's production-ready. The window where "I'm still evaluating" was a reasonable position has closed.

The decision tree:

Already on Spring Boot? → Spring AI 2.0. No contest. Annotation-driven, advisor-chain composable, MCP built in.
Want high-level goal-driven agents? → Embabel 1.0 (on top of Spring AI). GOAP replanning, typed domain objects.
Not on Spring? → LangChain4j 1.18. BDI pattern, broadest provider support, crash-resilient HITL.
Kotlin-first? → Koog 1.0. Multiplatform, type-safe DSL, A2A support.
Need managed deployment? → Spring AI AgentCore SDK. One annotation → managed runtime, memory, observability, scaling.

But the framework is 20% of the decision. The 80% is: Do you understand how MCP tool discovery works? Do you understand the 4 memory strategies? Do you understand where retry responsibility lives (infra vs reasoning)?

If yes — you can build production agents in any of these frameworks by next week.

If no — learning a second framework teaches you nothing new.

The full conversation goes deeper into each of these areas. Timestamps in the video description — jump to what matters most to you.

🎬 Watch: 68 Minutes That Could Change How You Think About Java and AI

📺 Subscribe: Code With Ease — By Varsha

📝 More writing: medium.com/@varshadas | dev.to/varshadas

🔗 Connect: linkedin.com/in/varsha-das-se

I Stopped Dragging Boxes in Draw.io (Here's What I Do Instead)

Varsha Das — Tue, 26 May 2026 13:46:37 +0000

If you're a Java developer, solutions architect, or anyone who's ever lost an afternoon to draw.io this one's for you.

Being part of 5 engineering teams over 8 years, here's something I experienced on almost every engineering team I've been part of. And you must have been too.

Product manager drops a PRD. We huddle in meeting rooms as devs with whiteboard markers flying, design discussions getting heated, someone sketching a system on the glass wall that actually makes sense. And then came the part everyone dreaded.

"Ok, now create a design doc and add the diagrams."

Design documents. Sequence diagrams. Class diagrams. Architecture diagrams. All of it formalized, version-controlled, and painstakingly created in draw.io.

I genuinely hated it.

And I think you know exactly what I mean. Dragging boxes. Aligning arrows. Snapping to grid. Unsnapping from grid because it snapped to the wrong thing. Spending 30 minutes on something or maybe more. It felt like the least productive version and the unglamorous part of engineering work and yet somehow it was always blocking the design review.

Honestly? I would have been happy to just take a photo of the whiteboard sketch and call it done. If only someone could magically understand it. Or if I could just speak out what I wanted to draw and have it appear.

I actually didn't mind sequence diagrams. The logic was satisfying. Mapping out the flow, seeing the interactions, watching the system tell its own story. I could get into that.

But then again with AWS architecture diagrams the problem wasn't really the icons.

If you've ever been responsible for architecture diagrams in a real team, you know exactly what I'm talking about. The pain is universal. And it's actually well-documented:

Creating professional AWS architecture diagrams is one of those tasks that sounds simple and never is. Solutions architects, developers, tech leads — everyone has to do it. And everyone has the same complaints.

It takes forever. The tools have a learning curve. draw.io, Lucidchart, Visio — they're not hard, but they're not fast either. And every new person on the team has to learn them from scratch.

Consistency is a constant battle. You make one diagram in one style, someone else makes another, and suddenly your documentation looks like it was designed by three different teams. Because it was.

AWS icons go stale. AWS releases new services, updates icon sets, renames things. Keeping your diagrams in sync with the official AWS visual language is a part-time job nobody signed up for.

And maintenance? Every time the architecture evolves and it always evolves you're back in the tool, reorganizing boxes, re-routing arrows, hoping nothing breaks the layout.

The result is that diagrams become a bottleneck. Or worse — they become outdated the moment they're published and nobody updates them because it's too painful.

So when I say I stopped dragging boxes — I mean I found a way to close that gap. To go from "system in my head" to "diagram on screen" without the tax in between.

Let me show you how.

There are two approaches I use — one for production-ready AWS architecture diagrams with official icons, and another for quick hand-drawn sketches when polish would feel premature. Let me show you both.

Part 1: Official AWS Diagrams with Kiro + MCP

Before we get into the setup, let me quickly explain what's actually happening under the hood — because understanding this makes everything click.

Kiro is an AI-powered IDE that brings generative AI capabilities directly into your development workflow.

MCP (Model Context Protocol)— developed by Anthropic as an open protocol — provides a standardized way to connect AI models to virtually any data source or tool. Think of it as a plugin system for AI. MCP servers act as specialized extensions that give Kiro domain-specific capabilities it wouldn't have on its own.

The two MCP servers we're using:

diagrams-mcp → generates diagrams using the Python diagrams package with the complete official AWS icon set
AWS Documentation MCP → searches and reads AWS documentation to validate best practices→ searches and reads AWS documentation to validate best practices before generating

Together, they give Kiro the ability to produce architecture diagrams that are both visually correct AND architecturally sound.

Setup (5 minutes, once)

Step 1: Install dependencies

# uv — a fast Python package/environment manager.
# The diagrams-mcp server runs as a Python tool via uvx (uv's package runner).
pip install uv

# Python 3.10 — required by the diagrams package for generating architecture PNGs.
# If you already have 3.10+ installed, skip this.
uv python install 3.10

# GraphViz — the layout engine that positions nodes and routes arrows in diagrams.
# Without it, the diagrams package can generate code but can't render images.
# macOS: brew install graphviz
# Ubuntu: sudo apt install graphviz
# Windows: choco install graphviz

Step 2: Configure MCP servers

Add this to ~/.kiro/settings/mcp.json:

{
  "mcpServers": {
    "aws-diagrams": {
      "command": "uvx",
      "args": ["diagrams-mcp"],
      "env": { "FASTMCP_LOG_LEVEL": "ERROR" },
      "autoApprove": [],
      "disabled": false
    },
    "aws-docs": {
      "command": "uvx",
      "args": ["awslabs.aws-documentation-mcp-server@latest"],
      "env": { "FASTMCP_LOG_LEVEL": "ERROR" },
      "autoApprove": [],
      "disabled": false
    }
  }
}

Kiro automatically discovers MCP servers from this file. That's it.

macOS note: If the servers fail to connect, uvx may not be in Kiro's PATH. Find your full path with which uvx in terminal and replace "uvx" with the full path (e.g. "/Users/yourname/.local/bin/uvx") in the config above.

Step 3: Verify the setup

Open the Kiro chat panel and check your MCP servers are connected from the MCP panel in the sidebar. Then test with a simple prompt:

"Please create a diagram showing an EC2 instance in a VPC connecting to an external S3 bucket. Include essential networking components (VPC, subnets, Internet Gateway, Route Table), security elements (Security Groups, NACLs), and clearly mark the connection between EC2 and S3. Label everything appropriately and indicate all resources are in us-east-1. Check AWS documentation to ensure it adheres to best practices before creating the diagram."

If you see a diagram, you're set up correctly.

What's happening when you run a prompt

When you describe what you want, here's the actual sequence:

Kiro searches AWS documentation for best practices using search_documentation
Reads the relevant docs using read_documentation
Lists the needed AWS service icons using list_icons
Generates Python code using the diagrams package
Executes it and returns a PNG

You describe what you want. The MCP servers handle the rest.

Final digram:

Real examples

Simple web app:

Create a diagram for a simple web application with an Application Load Balancer,
two EC2 instances, and an RDS database. Check AWS documentation to ensure it
adheres to best practices before creating the diagram.

Multi-tier architecture:

Create a diagram for a three-tier web application with a presentation tier
(ALB and CloudFront), application tier (ECS with Fargate), and data tier
(Aurora PostgreSQL). Include VPC with public and private subnets across
multiple AZs. Check AWS documentation for best practices.

Serverless:

Create a diagram for a serverless web application using API Gateway, Lambda,
DynamoDB, and S3 for static website hosting. Include Cognito for user
authentication and CloudFront for content delivery. Check AWS documentation
for best practices.

Data pipeline:

Create a diagram for a data processing pipeline with components organized
in clusters for data ingestion (Kinesis, SQS), processing (Lambda, Glue),
storage (S3, DynamoDB), and analytics (Athena, QuickSight). Check AWS
documentation for best practices.

And you iterate by just… talking to it:

"Add a WAF in front of CloudFront."
"Show DynamoDB Streams connecting to a Lambda for event processing."
"Make it multi-region with Route 53."

Each change takes seconds. Not 20 minutes of reorganizing boxes.

Part 2: Hand-Drawn Diagrams with Kiro Skills

Here's where it gets fun.

Sometimes you don't want a polished, corporate-looking diagram. Sometimes you want that whiteboard sketch feel — the kind you'd draw during a design discussion when everyone's still figuring things out.

Kiro has a hand-drawn-diagrams skill that generates Excalidraw-style sketchy diagrams. The aesthetic is intentional — it looks like a human drew it. Which makes it perfect for:

Blog posts (feels approachable, not intimidating)
Video explainers (you can animate it drawing itself)
Quick architecture discussions where polish would feel premature

Setup (one-time)

Download the skill zip and install it:

unzip ~/Downloads/hand-drawn-diagrams.zip -d ~/.kiro/skills/

Kiro picks it up automatically. No restart needed.

The prompt I used

Create a hand-drawn architecture diagram showing the MCP flow:

AI Agent → MCP Client → MCP Server → Spring Boot App → Amazon Bedrock

Layout: left-to-right flow
Style: hand-drawn sketch, monochrome
Shapes:
- AI Agent and Amazon Bedrock as ellipses (external actors)
- MCP Client, MCP Server, Spring Boot App as rectangles (services)

Label each arrow with the protocol:
- AI Agent → MCP Client: "tool call"
- MCP Client → MCP Server: "JSON-RPC"
- MCP Server → Spring Boot App: "HTTP/REST"
- Spring Boot App → Amazon Bedrock: "Bedrock API"

Add a short annotation below each node describing its role.
Add a title: "MCP Architecture Flow"

Open it in the Excalidraw editor.

What Kiro did

Routed the request to the hand-drawn-diagrams skill
Generated a full Excalidraw JSON with 24 elements, validated clean (0 errors)
Produced two live links instantly — no export, no download needed

👉 View & edit the diagram
— opens in Excalidraw, fully editable. Export PNG via hamburger menu → Export image → PNG.

👉 Watch it animate
— each node draws itself stroke by stroke. Perfect for screen recording as video content.

The animated version is genuinely great for explainer videos. Each node appears sequentially, arrows draw themselves, labels fade in. The kind of thing that would take hours in After Effects — done in one prompt.

Why This Actually Matters

This isn't just about saving time. Though it does — massively.

It's about removing friction from communication.

Architecture diagrams exist to explain systems to other humans. The faster you can go from "idea in your head" to "visual that others understand," the better engineer you become. The better communicator. The better collaborator.

And here's the thing I keep coming back to — MCP is the unlock. It's a standard protocol that lets AI tools connect to specialized capabilities.

Need AWS icons? MCP server for that.
Need best practices validation? MCP server for that.
Need hand-drawn aesthetics? Kiro skill for that.

The pattern is simple: describe what you want → get what you need.

TL;DR

What	How	Result
Official AWS diagrams	Kiro IDE + `diagrams-mcp`	Production-ready PNGs with correct icons
Same, from terminal	Kiro CLI + `diagrams-mcp`	Same output, no GUI needed
Best practices check	`aws-documentation-mcp-server`	Diagrams follow AWS Well-Architected
Hand-drawn sketches	Kiro `hand-drawn-diagrams` skill	Excalidraw-style, animatable diagrams
Iteration	Natural language follow-ups	Seconds per change, not hours

The SDLC pain point of "make a diagram" just became a one-liner.

If you're still dragging boxes in 2026 — try this. Your future self will thank you.

🔗 Reference: Build AWS architecture diagrams using Kiro CLI and MCP

What's the most painful diagram you've ever had to create? Drop it in the comments — I'll try generating it with a single prompt. 👇

AI Agents Don't Crash. They Drift. Here's the Framework to See It.

Varsha Das — Wed, 20 May 2026 18:03:41 +0000

The scariest AI agent failures don't trigger alerts. They look like success. Here's a 7-dimension resilience framework for building trust in agentic systems — based on the AWS Architecture Blog's approach to resilient generative AI agents.

💡 What this post covers: Why code getting cheap now creates a very big trust crisis, why the resilience patterns we have built for decades don't work for AI agents, and the 7-dimension framework I use to reason about trust in agentic systems.

A few months ago, a developer at the Summit told me a story that I haven’t stopped thinking about.

Her team had shipped an AI agent built in some 2 weeks, basically — that processed customer support tickets, classified them by urgency, and routed them to the right team. The demo was great. Stakeholders loved it. It went to production.

Two weeks later, someone on the receiving end asked:

"Hey, has something changed with the routing? I'm getting tickets that make no sense for my queue."

They checked their dashboards. Everything was green.

But something was wrong. The agent had been confidently routing tickets to the wrong teams for days.

Not all of them, just enough to confuse, but not enough to trigger an alarm.

That story just got me thinking so much that when I dug into it, there was no way of knowing how long the drift had been happening.

Yes, the drift, that itself is the main caveat.

The system looked healthy. The output was broken. And there was no framework for how to think or anticipate this kind of failure.

This blog is about that framework.

Let's dive right in……

The Tax on Ideas Just Hit Zero

For most of the history of software, there was and always has been a "significant" tax on ideas. You had an idea, and then you spent days or weeks or months turning it into working code.

The tax was high enough that most ideas died in a backlog.

You triaged ruthlessly.

You picked the three things that mattered most and let everything else pile up in the JIRA boards. (Much to the dismay of the Jira board owners, haha)

So that tax? It just hit zero.

AI agents can generate dozens of PRs overnight — building code, features, and entire systems. The gap between having an idea and seeing it built has effectively collapsed.

When code generation becomes nearly free, the bottleneck shifts:

from implementation to orchestration,

from writing to judgment,

from building to operating.

But here's what nobody tells you: when you can build code at the speed of thought, deploying that code to production becomes the bottleneck.

A system can be assembled at the speed of thought.

Trust is earned at a different pace entirely.

When Systems Fail Without Breaking

Last month, a Cursor agent deleted a company's entire production database.

This agent running Anthropic's Claude Opus 4.6 deleted PocketOS's entire production database — plus all backups — in nine seconds.

The agent was working on a routine task in a test environment when it hit a credentials problem. Instead of stopping, it found an API token in an unrelated file, a token that carried full account-wide permissions including destructive operations, and issued a single command that wiped everything.

No confirmation prompt. No warning. No check that it was targeting production instead of test.

Railway's backup model stored volume-level backups inside the same volume — so when the volume went, the backups went with it. The most recent recoverable backup was three months old.

When the founder asked the agent to explain itself, it produced what he called a "written confession": "I guessed instead of verifying. I ran a destructive action without being asked. I didn't understand what I was doing before doing it."

Two layers of guardrails — Cursor's published safety rules and the company's internal safety instructions — both told the agent not to do exactly what it did. Both failed at the same time.

The internet blamed the AI. But the real failure was an over-permissioned token sitting in a file the agent could read, paired with infrastructure that collapsed when the volume did.

That was a loud failure. Dramatic. Viral. Obvious.

The scarier ones? They're silent.

Consider an enterprise AI assistant designed to summarise regulatory updates for financial analysts. Every morning, this assistant retrieves documents from internal repositories, synthesizes them using a language model, and distributes summaries across internal channels. Technically, everything works.

But over time, something slips.

An updated document repository hasn't been added to the retrieval pipeline.

The assistant keeps producing summaries that are coherent and internally consistent — but they're increasingly based on obsolete information.

Nothing crashes. No alerts fire. Every component behaves as designed.

The problem is that the overall result is wrong.

From the outside, the system looks operational. All your monitoring dashboards read "healthy." Latency is fine. Error rates are zero.

Analysts are making decisions based on outdated regulatory information, and nobody knows. Catastrophic disaster for the business.

When humans wrote all the code, they at least understood what they shipped.

When agents generate it, the gap between "it works" and "I understand why it works" becomes the attack surface.

I've started calling these "green-dashboard failures." The kind where every metric says you're fine while the system is quietly betraying the people who depend on it.

Why the Patterns We Know Don’t Work Here

To understand why this is such a big threat, I need to take you back to how we've always built resilient systems.

Because the patterns we know, the ones we've built entire engineering practices around, they kind of break down here.

Over decades, we built resilience into three layers:

Infrastructure resilience. We deploy across multiple availability zones, auto-scale on demand, and load balance traffic — so if hardware fails, the system stays up.

Data resilience. We use read replicas, automated failover, and connection pooling — so if a database goes down, we don't lose data or availability.

Application resilience. We write circuit breakers, retry logic, and graceful degradation — so if a service fails, the app handles it predictably instead of crashing.

These patterns assume something fundamental: failures are binary.

A service is working or it's broken.

A sensor responds or it doesn't.

A constraint is met or it triggers a shutdown.

But AI agents don't crash. They degrade silently. They hallucinate confidently.

They might drift without a single metric turning red.

Autonomous Systems Behave Differently

While building and observing agentic systems for the past year, I see three things that make them fundamentally different from the software we've built for decades:

1. Continuous reasoning loop. They reason in loops, not steps. Unlike traditional request-response software, agents observe, think, and act in an ongoing cycle — always changing their own context.

2. Contextual inappropriateness. They produce output that is syntactically perfect but semantically wrong for the situation. A hallucinated paragraph looks like a real answer. A wrong tool call looks like a right one — until you trace what happened downstream.

3. Behavioral drift without errors. Small mistakes compound. The system gradually moves away from correct behaviour without any single step triggering an alarm.

It's not a cliff — it's a slow incline you don't notice until you're in the wrong valley.

This is why traditional resilience patterns break down.

So, we need a new framework.

The 7-Dimension Resilience Framework

Here's how we should think about building trust in agentic systems.

There are 7 dimensions you need to reason about and for each one, you ask: which failure modes apply here?

Foundation Models — Your LLM choice: self-hosted (you handle failover), managed or serverless. Each shifts resilience responsibility differently.

Something very basic like — If your model provider has a bad day, does your entire system go dark?
Agent Orchestration — The conductor. How agents coordinate, select tools, and escalate to humans. This is the brain — and if the brain makes a bad decision, the hands execute it perfectly.
Infrastructure — Where agents run: EC2, ECS, or a managed runtime like Bedrock AgentCore. If a container crashes, this layer handles the restart. The boring stuff that isn't boring when it fails.
Knowledge Base — Vector DBs, embeddings, RAG pipelines. If retrieval fails, your agent is answering questions without being able to look anything up. It doesn't know it's blind. It just confabulates.
Agent Tools — External dependencies: APIs, MCP servers, memory, prompt caching. What happens when that inventory API goes down? Does your agent wait forever, or does it move on?
Security & Compliance — Auth, guardrails, content validation. Prevents your agent from doing things it shouldn't — like leaking customer data or executing destructive actions without human approval.
Observability — Metrics, traces, reasoning logs. If you can't see why your agent made a decision, you can't fix it when it goes wrong.

That's the framework. 7 dimensions. Each one a surface where your agent can silently fail.

But knowing where things can break is only half the picture. The other half is knowing how they break — the specific failure modes, what they look like at 3 AM, and how to defend against each one.

In Part 2, I break down all 5 silent failure modes — with real-world case studies (including an agent that deleted a production database in 9 seconds) and the exact defenses for each.

This post is based on and extends the resilience framework from the AWS Architecture Blog: Build Resilient Generative AI Agents.

📺 More from the series:

Have you seen your agent "drift" without any metric catching it? How long before someone noticed? Drop it in the comments — I'll respond to every one.

🚀 How Developers Can Stop Pretending to Understand AI Buzzwords

Varsha Das — Thu, 11 Dec 2025 19:02:59 +0000

> If you can't explain it simply, you don't understand it well enough.
 - Albert Einstein

You know that feeling when someone starts talking about "agentic AI workflows with RAG pipelines and vector embeddings" and everyone's nodding like they totally get it? Yeah, I was that developer pretending to understand while feeling so lost within.

A few months ago, I hit my breaking point. Every dev thread, every tech talk—just buzzwords with zero actual clarity. So I stopped faking it and decided to actually learn things from scratch.

Plot twist: turns out most people are faking it too.

The Research Paper Rabbit Hole

My first move? Dive into IBM research papers. Because that's what good developers do, right? Don’t get me wrong — they’re good. Like, really good. I referred to them while writing this because of the kind of thorough, well-researched content that builds solid foundational knowledge. But, they’re dense. My brain started to explode after reading 1 paper.

Next stop: YouTube. Surely someone had figured out how to explain this without requiring a PhD? And yes, there's brilliant content out there. But here's the problem: you watch one video on transformers, another on embeddings, then someone casually mentions "attention mechanisms" and suddenly you’re like “wait, how does this connect to what I learned yesterday?”

I am pretty sure many of you would have been there.

And somewhere in the middle of all this chaos, I just thought: “Can someone PLEASE just give me one clean map? Like, all of it. In one place. That actually makes sense?”

So… I made one.

A humble attempt to build one, I would say.

Key Takeaways

You'll finally get a plain-talk view of AI terms that often feel too dense or too "expert-only."
You'll learn how the basics link together — models, prompts, safety, and the stuff that holds the whole AI stack in place.
You'll understand why prompts matter so much, why they sometimes go wrong, and what people do to keep them on track.
You'll get a sense of how machines learn, how they pull info, and how this leads to better answers.
You'll see the flow from simple chat systems to tools, tasks, and full-on AI helpers that can act on your behalf.
You'll be able to read AI threads, posts, papers or videos without feeling lost or drained.

So yeah… sit with a paper and pen, take notes if you want, maybe read this on your laptop, and slowly absorb. No rush.

Before We Start

If you're completely new, just make sure you've heard of these concepts:

Core Concepts

Neural Networks — The brain-inspired structure that powers modern AI, consisting of interconnected nodes that process information
Deep Learning — Using many layers of neural networks to learn complex patterns from large amounts of data
Natural Language Processing (NLP) — Teaching computers to understand, interpret, and generate human language
Machine Learning — The broader field where computers learn patterns from data without being explicitly programmed for every scenario
Training Data — The collection of examples used to teach AI models patterns and relationships
Model — The trained AI system that has learned patterns and can make predictions or generate outputs
Algorithm — The mathematical rules and procedures that guide how a model learns from data
Pattern Recognition — The ability of AI systems to identify recurring structures, relationships, and trends in data
Prediction — How trained models generate outputs by using learned patterns to make informed guesses about what comes next
Inference — The process of using a trained model to generate outputs or make decisions on new, unseen data

The Four Phase Journey

The Four-Phase Learning Framework

Instead of drowning in terminology, here's how AI concepts actually connect:

🎯 Phase 1: The Foundation — How AI Learns

First, the LLM must learn through training. This happens in three fundamental ways:

Supervised learning — labeled examples
Self-supervised learning — predicting missing pieces in unlabeled data (how modern LLMs are trained)
Reinforcement learning — trial-and-error with feedback

The Training Pipeline

During this training phase, the model processes massive amounts of text by:

Breaking it into tokens
Converting tokens into embeddings
Using attention mechanisms to understand which parts matter most
Building patterns across transformer layers that capture complex relationships

Tokenization breaks text into processable units, embeddings convert those tokens into numerical vectors in high-dimensional space, and self-attention mechanisms within transformer architectures determine which tokens matter most for context.

To make models production-ready, we use:

Distillation — shrinking big models into smaller ones
Quantization — reducing numerical precision from 32-bit to 8-bit or 4-bit to run faster on resource-constrained devices

🔍 Phase 2: Knowledge Retrieval — Bridging Training and Real-Time Access

Once trained, models need efficient ways to access information during inference. This is where semantic search and vector databases become critical.

How Semantic Search Works

Unlike traditional keyword matching, semantic search understands meaning:

Searching "smartphone" also retrieves "cellphone" and "mobile devices"
These concepts live close together in vector space

Vector Databases

Vector databases store data as high-dimensional numerical arrays, enabling lightning-fast similarity searches essential for real-time AI applications.

This retrieval capability forms the bridge between what models learned during training and what they can access when answering your questions—the foundation for everything that follows.

💬 Phase 3: User Interaction — Prompts, Safety, and Inference

Prompts are your interface for communicating with AI. When you submit a prompt, the model:

Tokenizes it
Converts tokens to embeddings
Generates responses one token at a time through inference
Calculates probabilities for potential next tokens
Outputs the most likely one

Prompt Engineering Techniques

Zero-shot — no examples provided
Few-shot — providing sample outputs
Chain-of-thought — step-by-step reasoning

Safety Considerations

However, prompts introduce risks:

Hallucinations — fabricated responses not grounded in training data
Prompt injection — malicious instructions disguised as user input

This is why guardrails—safeguards operating across data, models, applications, and workflows—are essential to keep AI systems safe, responsible, and within defined boundaries in production environments.

🤖 Phase 4: Advanced Applications — RAG, MCP, and Autonomous Agents

Now everything converges into autonomous systems.

RAG (Retrieval-Augmented Generation)

RAG solves the knowledge cutoff problem by:

Converting your question into vector embeddings
Performing semantic search across vector databases
Feeding retrieved information to the LLM as additional context

This enables AI to work with proprietary data or recent information without expensive retraining.

MCP (Model Context Protocol)

MCP provides the universal language for AI-tool communication, standardizing how agents:

Discover tools
Request data access
Execute actions safely
Receive results

Think of it like REST APIs but designed specifically for AI systems.

AI Agents & Multi-Agent Systems

AI Agents are autonomous systems that:

Break complex goals into subtasks (planning)
Use external tools to gather missing information (reasoning via RAG and MCP)
Make decisions and take actions independently
Learn from past interactions

Agentic AI is the umbrella term for this paradigm shift—AI that exhibits agency, acting independently rather than just answering questions.

Multi-agent systems represent the cutting edge: multiple specialized agents working together, each handling specific roles like input validation, business logic, data operations, and system monitoring.

🔄 The Complete Flow

Learn (Foundation) → Store (Vector Databases) → Retrieve (Semantic Search) → Apply (RAG + Prompts) → Act (MCP + Agents)

This isn't just terminology—it's the architectural pattern production AI systems are built on today, where models evolve from statistical predictors into autonomous problem-solvers that reason, plan, and execute complex workflows.

How Each Phase Connects

Phase 1 establishes the learning foundation (tokenization → embeddings → attention → transformers, plus the three learning types and optimization techniques)
Phase 2 bridges training to real-time access (semantic search and vector databases as the retrieval layer)
Phase 3 covers the interaction layer (prompts, inference, safety concerns, and guardrails)
Phase 4 brings it all together (RAG, MCP, and agents as the autonomous execution layer)

Each phase flows naturally into the next, creating a comprehensive understanding of how modern AI systems work from training to autonomous execution.

Stop Pretending. Start Understanding.

No more nodding along in meetings. No more feeling lost in AI discussions. Get the complete picture that connects every dot from tokens to autonomous agents.

This overview gives you the mental map, but each phase has layers of complexity that make the difference between "getting it" and actually understanding it.

I've written a 22-min comprehensive guide that breaks down every concept in more details and shows how they interconnect.

Read the full guide on Medium →

Stop pretending you understand AI buzzwords. Get the complete picture NOW!!

Found this helpful? Follow me for more developer-friendly AI content that actually makes sense.