DEV Community: Carlos Cortez 🇵🇪 [AWS Hero]

# Blog 10: Minecraft Mod Agent- I Made My AI Agent Judge Itself — And Used the Scores to Make It a Better Combat Advisor

Carlos Cortez 🇵🇪 [AWS Hero] — Sat, 01 Aug 2026 16:13:45 +0000

Blog 10: Minecraft Mod Agent- I Made My AI Agent Judge Itself — And Used the Scores to Make It a Better Combat Advisor

Cloud Swords Mod — Blog 10

In Blog 9, I gave my Minecraft mod an AI brain. A Strands Agent that decides how many allies to summon based on your combat situation. It works. But "works" isn't a metric.

How do I know the AI is making good decisions? When a player has 3 hearts and 8 zombies closing in, does the agent summon 5 allies (correct) or 1 (death sentence)? When the player is full health with no threats, does it correctly do nothing — or waste resources summoning minions for no reason?

I needed an evaluation framework. So I built one. And the judge? Another AI.

The Problem: Non-Deterministic Decisions

The agent uses gpt-oss:20b. Same input can produce slightly different outputs between calls. You can't write assert response == expected because there's no single correct answer.

What you CAN evaluate:

Is the action valid JSON? (deterministic check)
Is the action within bounds? (1-5 minions, valid buff types)
Is the response proportional to the threat? (requires judgment)
Is the reasoning coherent? (requires judgment)

The first two are code. The last two need an LLM.

The Eval Dataset: Real Interactions

Remember interactions.json from the mock server? Every time the agent makes a decision, it's logged:

{
  "timestamp": "2026-05-19T19:17:34",
  "payload": {
    "player": "Carlos",
    "weapon": "sword_of_lambda",
    "health": 5,
    "nearby_mobs": 10,
    "biome": "nether",
    "dimension": "the_nether"
  },
  "decision": {
    "action": "summon",
    "count": 5,
    "duration": 10,
    "reason": "low health, many mobs"
  },
  "source": "agent"
}

I took 20+ real interactions and added expected behavior ranges:

eval_dataset = [
    {
        "context": {"health": 5, "nearby_mobs": 10, "biome": "nether", "dimension": "the_nether"},
        "expected_action": "summon",
        "expected_count_range": [4, 5],
        "expected_reasoning": "Should recognize critical threat (low HP + many mobs + dangerous biome)",
        "tags": ["critical", "nether"]
    },
    {
        "context": {"health": 20, "nearby_mobs": 0, "biome": "plains", "dimension": "overworld"},
        "expected_action": "none",
        "expected_count_range": [0, 0],
        "expected_reasoning": "Full health, no threats — should do nothing",
        "tags": ["safe", "overworld"]
    },
    {
        "context": {"health": 12, "nearby_mobs": 3, "biome": "dark_forest", "dimension": "overworld"},
        "expected_action": "summon",
        "expected_count_range": [1, 3],
        "expected_reasoning": "Moderate threat — proportional response",
        "tags": ["moderate", "overworld"]
    },
    # ... 17 more scenarios
]

Level 1: Code Evaluators (Free, Instant)

def eval_valid_json(response: str) -> bool:
    """Can we parse the response as JSON?"""
    try:
        json.loads(response)
        return True
    except:
        return False

def eval_action_in_bounds(decision: dict) -> bool:
    """Is the action within allowed bounds?"""
    action = decision.get("action")
    if action == "summon":
        count = decision.get("count", 0)
        duration = decision.get("duration", 0)
        return 1 <= count <= 5 and 5 <= duration <= 15
    elif action == "buff":
        valid_effects = {"regeneration", "resistance", "strength", "speed"}
        return decision.get("effect") in valid_effects and 3 <= decision.get("duration", 0) <= 10
    elif action == "none":
        return True
    return False

def eval_has_reason(decision: dict) -> bool:
    """Does the decision include a reason?"""
    reason = decision.get("reason", "")
    return len(reason) > 3 and len(reason) < 50

def eval_proportional(decision: dict, context: dict, expected: dict) -> bool:
    """Is the response roughly proportional to the threat?"""
    if expected["expected_action"] == "none" and decision.get("action") == "none":
        return True
    if expected["expected_action"] == "summon":
        count = decision.get("count", 0)
        low, high = expected["expected_count_range"]
        return low <= count <= high
    return decision.get("action") == expected["expected_action"]

These catch 60% of problems instantly: malformed JSON, out-of-bounds values, missing reasons, wildly disproportionate responses.

Level 2: LLM-as-Judge

For the subjective stuff — "is this a good decision?" — I use the same model as a judge:

JUDGE_PROMPT = """You are evaluating an AI combat advisor for a Minecraft mod.

Given the player context and the AI's decision, score it on:
1. **Threat Assessment** (1-10): Did the AI correctly gauge the danger level?
2. **Proportionality** (1-10): Is the response scaled to the threat?
3. **Reason Quality** (1-10): Is the reasoning clear and accurate?

Context: {context}
AI Decision: {decision}
Expected behavior: {expected}

Respond ONLY with JSON:
{{"threat_assessment": X, "proportionality": X, "reason_quality": X, "feedback": "one sentence"}}"""

def judge_decision(context: dict, decision: dict, expected: dict) -> dict:
    prompt = JUDGE_PROMPT.format(
        context=json.dumps(context),
        decision=json.dumps(decision),
        expected=expected["expected_reasoning"]
    )
    result = agent(prompt)
    return json.loads(str(result))

The meta part: the same AI model that makes the decisions is also judging them. This works because the judge has the expected behavior as reference — it's not evaluating in a vacuum.

The Eval Runner

class CombatAdvisorEval:
    def __init__(self, agent):
        self.agent = agent
        self.results = []

    def run(self, dataset: list[dict]) -> dict:
        for case in dataset:
            # Get agent decision
            decision = decide(self.agent, case["context"])

            # Level 1: Code checks
            checks = {
                "valid_json": eval_valid_json(json.dumps(decision)),
                "in_bounds": eval_action_in_bounds(decision),
                "has_reason": eval_has_reason(decision),
                "proportional": eval_proportional(decision, case["context"], case),
            }

            # Level 2: LLM Judge
            judge_scores = judge_decision(case["context"], decision, case)

            self.results.append({
                "context": case["context"],
                "decision": decision,
                "checks": checks,
                "judge_scores": judge_scores,
                "tags": case["tags"],
            })

        return self._summary()

    def _summary(self) -> dict:
        total = len(self.results)
        return {
            "total_cases": total,
            "code_pass_rate": sum(
                all(r["checks"].values()) for r in self.results
            ) / total,
            "avg_threat_assessment": sum(
                r["judge_scores"]["threat_assessment"] for r in self.results
            ) / total,
            "avg_proportionality": sum(
                r["judge_scores"]["proportionality"] for r in self.results
            ) / total,
            "avg_reason_quality": sum(
                r["judge_scores"]["reason_quality"] for r in self.results
            ) / total,
        }

A/B Testing System Prompts

The real power: I can test different system prompts and measure which one produces better decisions.

PROMPT_A = """You are a combat advisor. Given context, decide one action..."""  # Original

PROMPT_B = """You are a combat advisor. IMPORTANT: Always consider the biome danger level.
Nether and End are inherently more dangerous than Overworld.
Scale your response accordingly..."""  # Biome-aware variant

# Run eval with both
agent_a = Agent(model=model, system_prompt=PROMPT_A)
agent_b = Agent(model=model, system_prompt=PROMPT_B)

results_a = CombatAdvisorEval(agent_a).run(eval_dataset)
results_b = CombatAdvisorEval(agent_b).run(eval_dataset)

print(f"Prompt A — proportionality: {results_a['avg_proportionality']:.1f}/10")
print(f"Prompt B — proportionality: {results_b['avg_proportionality']:.1f}/10")

Prompt B scored higher on Nether scenarios because it explicitly considers biome danger. That insight came from the eval — not from vibes.

The Feedback Loop

Observe (interactions.json)
    ↓
Evaluate (code checks + LLM judge)
    ↓
Adjust (modify system prompt)
    ↓
Measure (re-run eval, compare scores)
    ↓
Deploy (update prompt in mock server)
    ↓
Observe again...

This is the same loop from the LLMOps series (Part 3: Eval-Driven Development) — but applied to a Minecraft mod. The pattern is universal.

Results: What I Found

After running evals across 20 scenarios:

Metric	Prompt v1	Prompt v2 (biome-aware)
Code pass rate	95%	95%
Threat assessment	7.2/10	8.4/10
Proportionality	6.8/10	8.1/10
Reason quality	7.5/10	7.8/10

The biome-aware prompt improved threat assessment by 1.2 points — specifically in Nether and End scenarios where v1 was under-responding.

The 5% code failure rate? JSON parsing issues when the model occasionally adds explanation text before the JSON. Fixed by making the parser more robust (find first {, last }).

Cloud/AI Concepts Taught

Eval Concept	What It Teaches
Code evaluators	Input validation, schema enforcement
LLM-as-Judge	AI evaluating AI (meta-evaluation)
Eval dataset	Test cases for non-deterministic systems
A/B testing prompts	Experimentation, data-driven decisions
Feedback loop	Continuous improvement, observability
Scoring thresholds	Quality gates, SLOs for AI
interactions.json	Observability, audit trails

What I'd Do Next

Automated eval in CI — every prompt change triggers eval, blocks if scores drop
Player feedback — thumbs up/down after each cloud invocation, feeds back into dataset
Seasonal prompts — different system prompts for different game phases (early/mid/endgame)
Multi-model comparison — test gpt-oss:20b vs Claude Haiku vs Nova Lite for cost/quality tradeoff

The Full Picture

Across 10 blog posts, this mod went from "7 swords with cloud names" to a full system with:

Custom textures generated with Python + AI
Energy systems with BFS algorithms
Multiblock machines with custom GUIs
A spell system with 28 spells
Armor sets with dodge mechanics
Wandering merchants with elite progression
A serverless AI backend
An evaluation framework for AI decisions

All teaching cloud computing through gameplay. All open source.

This concludes the Cloud Swords blog series. 10 posts, one mod, and more cloud concepts than most certification courses. Thanks for reading.

Code:

I'm Carlos Cortez — and this is Breaking the Cloud.

LinkedIn · GitHub · Dev.to

# Blog #9: I Gave My Minecraft Mod a Serverless Brain — And It Decides How Many Allies to Summon Based on How Screwed You Are

Carlos Cortez 🇵🇪 [AWS Hero] — Sat, 01 Aug 2026 16:13:30 +0000

I Gave My Minecraft Mod a Serverless Brain — And It Decides How Many Allies to Summon Based on How Screwed You Are

Cloud Swords Mod — Blog 9

I've been building a Minecraft mod that teaches cloud computing through gameplay. Swords named after AWS services. Energy systems that work like network infrastructure. Villagers that trade cloud certifications.

But version 0.3.8 is where things got weird. I connected the mod to an actual AI agent running in the cloud. Now when you swing the Sword of Lambda, instead of spawning a fixed number of minions — an AI evaluates your combat situation and decides how many allies you need.

The game never freezes. The AI never breaks balance. And the whole thing runs on a serverless backend that costs essentially nothing.

Here's how I built it.

The Items That Make It Work

Tome of API Key

Every cloud connection starts with authentication. In the mod, that's the Tome of API Key — a rare artifact found in dungeon chests (8% drop rate in bastions, end cities, nether fortresses).

When you first pick it up, it generates a unique API key (csk-XXXX format) and shows 100 charges. It glows with enchant shimmer. It's your passport to the cloud.

// On first inventory tick, generate unique key
if (!stack.getNbt().contains("api_key")) {
    String key = "csk-" + UUID.randomUUID().toString().substring(0, 8);
    stack.getNbt().putString("api_key", key);
    stack.getNbt().putInt("charges", 100);
}

The tooltip shows your key preview, remaining charges, and connection status. When charges hit zero, the cloud refuses your invocations — just like hitting a rate limit on a real API.

Cloud API Requester (Block)

The Cloud API Requester is the bridge between Minecraft and AWS. It's a crafted block (Iron + Circuit Board + Redstone + Ender Pearl + Data Core) with two slots:

Your Tome of API Key
The weapon you want to cloud-enable

Drop both in, wait ~8 seconds (the "deployment timer"), and your weapon gets marked with cloud_enabled=true in its NBT data. The tooltip now shows "☁ Cloud Enabled" in cyan.

This is the CI/CD pipeline of the mod. You're deploying your weapon to the cloud.

Cloud-Enabled Weapons

Once cloud-enabled, the Sword of Lambda changes behavior completely. Instead of spawning a hardcoded number of minions, it:

Sends your combat context to the backend (health, nearby mobs, biome, dimension)
Shows "☁ Cloud invocation sent..." in the actionbar
Waits for the AI's decision
Applies it: "☁ Cloud decided: 5 minions (Critical threat — max invocations)"

If you're NOT cloud-enabled, the sword works exactly as before. Backward compatible. No cloud required.

The Architecture

┌─────────────────────────────────────────────────────────────┐
│                     MINECRAFT CLIENT                         │
│                                                             │
│  Player swings Sword of Lambda (cloud-enabled)              │
│       │                                                     │
│       ▼                                                     │
│  CloudCombatHandler builds context:                         │
│  {player, weapon, health, biome, dimension, nearby_mobs}   │
│       │                                                     │
│       ▼                                                     │
│  CloudApiClient.sendEvent(context)  ← async, non-blocking  │
└───────┬─────────────────────────────────────────────────────┘
        │ POST /event
        ▼
┌─────────────────────────────────────────────────────────────┐
│                    BACKEND (CDK Stack)                        │
│                                                             │
│  API Gateway (rate limited: 100/day, 10/sec)                │
│       │                                                     │
│       ▼                                                     │
│  Lambda: IngestEvent                                        │
│  - Receives context                                         │
│  - Calls Strands Agent (or rule-based fallback)             │
│  - Stores decision in DynamoDB (TTL: 60s)                   │
│  - Returns {request_id}                                     │
│                                                             │
│  DynamoDB: CloudSwordsResults                               │
│  - Partition key: request_id                                │
│  - TTL auto-cleanup                                         │
│                                                             │
│  Lambda: PollResult                                         │
│  - GET /result/{request_id}                                 │
│  - Returns decision when ready                              │
└───────┬─────────────────────────────────────────────────────┘
        │ Poll every 2s
        ▼
┌─────────────────────────────────────────────────────────────┐
│  CloudApiClient receives decision                           │
│       │                                                     │
│       ▼                                                     │
│  CloudCombatHandler.applyDecision()                         │
│  - Summon "☁ Cloud Invocation" minions (named, cyan)        │
│  - Or apply buff (regeneration, resistance, strength)       │
│  - Or do nothing ("No action needed")                       │
└─────────────────────────────────────────────────────────────┘

The Async Pattern: Fire-and-Forget + Polling

The game can NEVER freeze waiting for a network response. So I used the same pattern you'd use in any real-time application:

Fire: Send the event, get back a request_id immediately
Forget: Continue gameplay, don't block
Poll: Every 2 seconds (via ServerTickEvents), check if the result is ready
Apply: When ready, queue the callback for the server thread

// CloudApiClient — non-blocking
public void sendEvent(JsonObject context) {
    httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofString())
        .thenAccept(response -> {
            String requestId = parseRequestId(response.body());
            startPolling(requestId);
        });
}

// Polling on server tick (every 40 ticks = 2 seconds)
ServerTickEvents.END_SERVER_TICK.register(server -> {
    // Execute queued callbacks on server thread
    Runnable callback;
    while ((callback = pendingCallbacks.poll()) != null) {
        callback.run();
    }
});

The threading fix was critical. Java's HttpClient runs callbacks on its own async thread — but Minecraft entity spawning MUST happen on the server thread. Without the ConcurrentLinkedQueue bridge, buffs wouldn't apply and entities would crash the game.

The AI Agent: Strands + Ollama Cloud

The brain is a Strands Agent running gpt-oss:20b via Ollama Cloud (free tier — no Bedrock cost for development):

from strands import Agent
from strands.models.ollama import OllamaModel

SYSTEM_PROMPT = """You are a combat advisor for a Minecraft RPG mod.
Given player context, decide ONE action.

Available actions (respond ONLY with valid JSON):
- {"action": "summon", "count": 1-5, "duration": 5-15, "reason": "brief"}
- {"action": "buff", "effect": "regeneration|resistance|strength|speed", "duration": 3-10, "reason": "brief"}
- {"action": "none", "reason": "brief"}

Rules:
- If health < 8 and many mobs: summon max allies or buff resistance
- If health > 15 and few mobs: action none or summon 1
- Scale response to threat level
- ONLY output the JSON object, nothing else"""

model = OllamaModel(
    host="https://api.ollama.com",
    model_id="gpt-oss:20b",
)
agent = Agent(model=model, system_prompt=SYSTEM_PROMPT)

The agent receives context like:

{
  "player": "Carlos",
  "weapon": "sword_of_lambda",
  "health": 6,
  "nearby_mobs": 8,
  "biome": "nether_wastes",
  "dimension": "minecraft:the_nether"
}

And responds:

{"action": "summon", "count": 5, "duration": 10, "reason": "Critical threat"}

The system prompt enforces bounded actions — the AI can't break game balance because it can only choose from a fixed set of actions with capped values.

The CDK Stack

The production backend is a single CDK stack:

class CloudSwordsStack(Stack):
    def __init__(self, scope, id, **kwargs):
        super().__init__(scope, id, **kwargs)

        # DynamoDB for async results
        results_table = dynamodb.Table(self, "DecisionResults",
            partition_key={"name": "request_id", "type": "STRING"},
            time_to_live_attribute="ttl",
            billing_mode=dynamodb.BillingMode.PAY_PER_REQUEST,
        )

        # Lambda: receive event, decide, store
        ingest_fn = _lambda.Function(self, "IngestEvent",
            runtime=_lambda.Runtime.PYTHON_3_12,
            handler="handler.lambda_handler",
            code=_lambda.Code.from_asset("lambda/ingest"),
        )

        # Lambda: poll result
        poll_fn = _lambda.Function(self, "PollResult",
            runtime=_lambda.Runtime.PYTHON_3_12,
            handler="handler.lambda_handler",
            code=_lambda.Code.from_asset("lambda/poll"),
        )

        # API Gateway with usage plan (rate limiting = gameplay mechanic)
        api = apigw.RestApi(self, "CloudSwordsApi")
        plan = api.add_usage_plan("Basic",
            throttle={"rate_limit": 10, "burst_limit": 20},
            quota={"limit": 100, "period": apigw.Period.DAY},
        )

The usage plan isn't just infrastructure — it's a gameplay mechanic. 100 requests per day means you can't spam the cloud. You have to be strategic about when you invoke it. Just like real API budgets.

Local Development: The Mock Server

For development, I run a mock server that integrates the Strands Agent directly:

# mock_server.py — localhost:8777
agent = None
try:
    from combat_advisor import create_agent, decide
    agent = create_agent()
    print("✓ Strands Agent loaded")
except:
    print("⚠ Agent unavailable, using rule-based fallback")

def decide_action(body):
    if agent:
        return decide(agent, body)
    # Fallback: simple rules
    if body["health"] < 8 and body["nearby_mobs"] > 5:
        return {"action": "summon", "count": 5, "duration": 10, "reason": "Critical"}
    return {"action": "summon", "count": 1, "duration": 5, "reason": "Standard"}

The mock server logs every interaction to interactions.json — which becomes the eval dataset for Blog 10.

Cloud Concepts Taught Through Gameplay

Game Mechanic	Cloud Concept
Tome of API Key	API keys + authentication
100 charges	Rate limiting / usage plans
Cloud API Requester (8s timer)	CI/CD deployment pipeline
`cloud_enabled=true` NBT	Feature flags
POST /event → request_id	Async event processing
Poll every 2s	Eventual consistency
AI decides action	Serverless compute (Lambda)
Bounded actions (1-5 minions)	Guardrails / IAM boundaries
CDK stack	Infrastructure as Code
Mock server	Local development environment

What I Learned

Threading in Minecraft is brutal — anything touching entities must be on the server thread. Async HTTP + game logic = careful queue management.
Rate limiting as gameplay works — players naturally strategize around limited cloud invocations. It teaches resource management without a lecture.
AI guardrails through system prompts — bounding the action space (1-5 minions, specific buffs) means the AI literally cannot break the game. The worst it can do is summon 5 minions when 3 would suffice.
Fallback patterns matter — if the cloud is down, the sword still works with local logic. Graceful degradation, taught through a video game.

What's Next

The AI makes decisions. But are they good decisions? In Blog 10, I build an evaluation framework that judges the agent's combat advice — using another AI as the judge. Meta? Absolutely. Useful? Surprisingly yes.

Code:

I'm Carlos Cortez — cloud engineer, gamer, and apparently someone who connects Minecraft to AI agents for fun. This is Breaking the Cloud.

LinkedIn · GitHub · Dev.to

I Built a Cloud Caravan Merchants Progression System as a cloud Marketplace

Carlos Cortez 🇵🇪 [AWS Hero] — Sat, 01 Aug 2026 16:10:01 +0000

Every cloud platform has a marketplace. AWS has the AWS Marketplace, Azure has its own, GCP too. They're places where you find specialized vendors offering services you can't build yourself — and the more you buy, the more they offer you.

We built that in Minecraft: Cloud Caravan Merchants. Three specialized traders that travel with donkeys carrying loot chests, offer 5 base trades each, and evolve into Elite merchants when you complete their missions — unlocking premium trades including Elytra, Tridents, and Totems of Undying.

🎮 The learning angle: The Elite progression system mirrors how cloud marketplace vendors work — start with basic offerings, prove you're a serious customer (complete trades), and unlock enterprise-tier products. The 3 merchant types map to cloud specializations: infrastructure (Packet Runner), platform (Protocol Broker), and application layer (Quantum Courier).

3 Merchants — 3 Cloud Layers

Merchant	Color	Specialty	Cloud Layer
Packet Runner	Cyan	Raw materials & resources	Infrastructure (IaaS)
Protocol Broker	Orange	Equipment & upgrades	Platform (PaaS)
Quantum Courier	Purple	Runes, tomes & magic	Application (SaaS)

Each spawns with a donkey carrying a loot chest with starter items. It's a welcome package — like free tier credits when you sign up for a cloud platform.

The Spawn Egg — Full Caravan Unit

The custom spawn egg doesn't just spawn a merchant. It spawns the entire unit:

Merchant entity (custom type, wandering trader model)
Donkey (tamed, with chest)
Loot in the chest (via loot table)
Leash connecting donkey to merchant

@Override
public ActionResult useOnBlock(ItemUsageContext context) {
    // 1. Spawn merchant
    CloudMerchantEntity merchant = merchantType.create(world);
    merchant.setPosition(pos.getX() + 0.5, pos.getY(), pos.getZ() + 0.5);
    world.spawnEntity(merchant);

    // 2. Spawn donkey with chest
    DonkeyEntity donkey = EntityType.DONKEY.create(world);
    donkey.setTame(true);
    donkey.setHasChest(true);

    // 3. Fill chest via NBT (inventory must be properly sized)
    NbtCompound donkeyNbt = new NbtCompound();
    donkey.writeNbt(donkeyNbt);
    donkeyNbt.putBoolean("ChestedHorse", true);
    NbtList itemsList = new NbtList();
    // ... add loot items to NBT
    donkey.readNbt(donkeyNbt);
    world.spawnEntity(donkey);

    // 4. Leash
    donkey.attachLeash(merchant, true);
}

The Donkey Chest Bug — A War Story

Fun debugging story: donkeys in 1.20.1 have a SimpleInventory field called items. When you call setHasChest(true), it's supposed to expand the inventory from 2 slots (saddle + armor) to 17 slots (+ 15 chest slots). But the expansion happens in onChestedStatusChanged(), which recreates the inventory.

If you try to put items in the chest BEFORE the inventory is properly sized, you get ArrayIndexOutOfBoundsException. If you put them AFTER spawning, the inventory might not be synced to the client yet.

The solution: write the chest contents via NBT. Set ChestedHorse: true and Items: [...] in the NBT compound, then call readNbt(). This forces the entity to reconstruct its inventory with the correct size AND the items already in place.

Elite Progression — The Loyalty Program

Complete all 5 base trades (use them until sold out) and the merchant evolves:

Base (5 trades) → Elite ★ (7 trades) → Elite ★★ (9 trades) → Elite ★★★ (11 trades)

Each promotion:

Adds 2 new trades with premium items
Changes the merchant's name to gold with stars
Triggers a Totem of Undying particle burst
Adds ambient END_ROD particles

private void checkElitePromotion() {
    int requiredTrades = 5 + (eliteLevel * 2);
    if (completedTrades >= requiredTrades && eliteLevel < 3) {
        eliteLevel++;
        applyEliteName(); // "★★ Protocol Broker ★★"
        addEliteTrades(this.getOffers(), eliteLevel);
        // Celebration particles
        world.spawnParticles(ParticleTypes.TOTEM_OF_UNDYING, ...);
    }
}

Elite Trades — Premium Vanilla Items

Cycle	Packet Runner	Protocol Broker	Quantum Courier
★	Quantum Shards, Spyglass	Netherlink Plating, Enchanted Golden Apple	Deployed Runes, Ender Eyes
★★	Cloud Edge Pearls, Echo Shards	Cloudsteel Plating, Trident	Grimoire of Architect, XP Bottles ×16
★★★	Nether Star, Totem of Undying	Netherite ×2, Elytra	Quantum Runes ×16, End Crystals

The endgame trades offer items that are normally very hard to obtain (Elytra, Trident) in exchange for Cloud Credits and rare materials. It's the cloud marketplace premium tier.

Natural Spawning — The Caravan Arrives

Every ~1 hour of real time, there's a 15% chance a caravan spawns 48-96 blocks from a random player:

private static void spawnCaravan(ServerWorld world, BlockPos pos, Random random) {
    int caravanSize = 1 + random.nextInt(3); // 1, 2, or 3 merchants
    // Shuffle types to avoid duplicates
    List<MerchantType> available = new ArrayList<>(List.of(MerchantType.values()));
    Collections.shuffle(available);

    for (int i = 0; i < caravanSize; i++) {
        // Spawn merchant + donkey at offset positions
        spawnUnit(world, pos, available.get(i), i * 2.5);
    }
}

A full caravan (3 merchants) gives you access to all specializations at once. It's like a cloud conference — all the vendors in one place.

Never Despawn — Persistent Infrastructure

Unlike vanilla wandering traders that disappear after 40 minutes, Cloud Merchants are permanent:

this.setPersistent();
this.setDespawnDelay(Integer.MAX_VALUE);

Once you find (or spawn) a merchant, they stay forever. They're your infrastructure — you invested in them, they stick around.

Custom Textures — Visual Identity

Each merchant uses a recolored wandering trader texture generated with Python PIL:

def recolor(data, hue_shift):
    """Shift hue of non-skin pixels"""
    for pixel in data:
        if not is_skin_tone(pixel):
            h, s, v = rgb_to_hsv(pixel)
            h = (h + hue_shift) % 1.0
            pixel = hsv_to_rgb(h, s, v)

Packet Runner: cyan shift (networking)
Protocol Broker: orange shift (fire/security)
Quantum Courier: purple shift (quantum/magic)

Rune Table Recipes — Mod Interop

We also added recipes for the Rune Crafting Altar (from the Runes mod):

Rune	Altar Input	Output	Savings
Packet	Cloud Shard + Redstone	×2	-1 Lapis, +1 output
Protocol	Packet Rune + Blazing Shard	×2	-1 Gold, +1 output
Quantum	Protocol Rune + Quantum Shard	×2	-1 Ender Pearl, +1 output

This is mod interop — using another mod's crafting system to provide alternative recipes. Like using a third-party service that's cheaper than building your own.

Lessons Learned

Challenge	Solution
Donkey chest inventory size	Write items via NBT, not direct slot access
Entity renderer missing = crash	Register renderer in ClientModInitializer
Wandering trader despawn timer	Override with MAX_VALUE despawnDelay
Spawn egg model missing	`template_spawn_egg` parent model
Trade completion tracking	NBT persistence (CompletedTrades, EliteLevel)

Series Wrap-Up

Over 8 posts, we went from "what if AWS services were Minecraft swords" to a full tech-magic mod with:

7 cloud-themed swords with unique abilities
28 spells across 9 spellbooks
4 armor sets with set bonuses
Multiblock machines with BFS energy networks
7 villager professions with a Cloud Credits economy
3 Caravan Merchants with Elite progression
Full compatibility with 650+ mod modpacks

The mod teaches cloud computing by making you experience the concepts. You don't read about Auto Scaling — you feel your damage stack. You don't study fault tolerance — you dodge attacks with Quantum Armor. You don't memorize service tiers — you progress through them.

That's the power of game-based learning. And we built it all with AI as a co-pilot.

The complete mod is on GitHub. Clone it, build it, play it.

Connect with me:

I'm Carlos Cortez, this is Breaking the Cloud, and this series proved that the best way to learn cloud is to play it. See you in the next adventure! 🎮☁️⚔️

Designing an Armor Progression That Mirrors Cloud Certification Levels

Carlos Cortez 🇵🇪 [AWS Hero] — Sat, 01 Aug 2026 16:09:22 +0000

In cloud computing, you level up through certifications: Practitioner → Associate → Professional → Specialty. Each level unlocks more capabilities and proves deeper expertise. We built the same thing as armor sets in Minecraft.

4 armor sets, each with a set bonus, a dodge mechanic, and a clear progression path that mirrors going from cloud beginner to architect.

🎮 The learning angle: Dataflow Armor gives Speed (network optimization). Netherlink gives Fire Resistance (firewall protection). Quantum gives damage resistance (fault tolerance). Cloudsteel gives flight (cloud-native freedom). The set bonuses ARE the cloud concepts.

4 Armor Sets — 4 Cloud Maturity Levels

Set	Tier	Armor	Toughness	Cloud Level
Dataflow	Post-Iron	18	1.5	Practitioner — basic networking
Netherlink	Post-Diamond	20	2.5	Associate — security focus
Quantum	Post-Netherite	24	3.5	Professional — fault tolerance
Cloudsteel	Endgame	24	4.0	Specialty — full mastery

Each set has a unique visual identity:

Dataflow: Cyan/teal with circuit line patterns + bright nodes
Netherlink: Dark red/orange with ember glow edges
Quantum: Purple with glitch pixels (digital distortion)
Cloudsteel: White/silver with cloud wisps + luminous edges

Set Bonuses — Cloud Capabilities

When you wear all 4 pieces of a set, you get passive effects:

Set	Effects	Dodge %	Cloud Concept
Dataflow	Speed I + Haste I	10%	Network optimization — faster throughput
Netherlink	Fire Resistance + Strength I (Nether)	13%	Firewall — immune to fire attacks
Quantum	Resistance I	15%	Fault tolerance — reduced impact
Cloudsteel	Slow Falling + Jump Boost II	20%	Cloud-native — freedom of movement

The Dodge System

Every set has a chance to completely nullify incoming damage. When triggered:

Portal particles burst around the player
Enderman teleport sound plays
Damage is set to zero

public class QuantumDodgeHandler {
    public static void register() {
        // ALLOW_DAMAGE event — return false to cancel damage
        DamageEvents.ALLOW.register((entity, source, amount) -> {
            if (!(entity instanceof PlayerEntity player)) return true;

            float dodgeChance = getSetDodgeChance(player);
            if (dodgeChance > 0 && player.getRandom().nextFloat() < dodgeChance) {
                // Dodge! Cancel damage
                spawnDodgeParticles(player);
                return false;
            }
            return true;
        });
    }
}

Higher-tier armor = higher dodge chance. It's probabilistic fault tolerance — like how multi-AZ deployments reduce (but don't eliminate) the chance of downtime.

Crafting Progression — The Certification Path

You don't jump straight to Cloudsteel. The progression uses the Smithing Table:

Iron Armor + Dataflow Plating + Cloud Shard → Dataflow Armor
Diamond Armor + Netherlink Plating + Blazing Shard → Netherlink Armor
Netherite Armor + Quantum Plating + Quantum Shard → Quantum Armor
Quantum Armor + Cloudsteel Plating + Cloud Edge Pearl → Cloudsteel Armor

Each tier requires the previous tier's armor as base. You can't skip levels — just like certifications build on each other.

Platings (4 ingots each)

III
I_I  →  1 Plating
___

Simple shaped recipe. 4 platings per set (helmet, chest, legs, boots) × 4 ingots = 16 ingots per full set.

Accessories Integration — Optional Dependencies

The mod works with or without the Accessories mod installed. When present, the Cloud Engineer Lens can be equipped in the face trinket slot instead of taking the helmet slot.

public class AccessoriesCompat {
    public static boolean isEquipped(PlayerEntity player, Item item) {
        // Runtime check — only call if mod is loaded
        var component = AccessoriesCapability.get(player);
        if (component == null) return false;
        return component.isEquipped(item);
    }
}

Detection priority: Accessories slot → Trinkets slot → Helmet → Offhand → Mainhand.

This is the optional dependency pattern — your mod works standalone, but integrates with other mods when available. Like how a cloud service works alone but integrates with others via APIs.

Spell Book Slot — Cross-System Integration

All 18 tomes (9 base + 9 empowered) implement a SpellBookItem interface and register in the trinkets:charm/spell_book tag. This means they can be equipped in the Spell Book accessory slot from other magic mods.

Cross-mod compatibility isn't just nice to have — it's essential for modpack inclusion. Same as cloud services: if your service doesn't integrate with the ecosystem, nobody uses it.

Nether Star Duplication — Endgame Recipe

4 Cloud Edge Pearls + 4 Data-Infused Quantum Shards + 1 Nether Star → 2 Nether Stars

An endgame recipe that avoids Wither farming. Expensive inputs, but saves time. It's like Reserved Instances — pay more upfront in materials, save time in the long run.

Lessons Learned

Challenge	Solution
Set bonus detection every tick is expensive	Check every 20 ticks (1 second), not every tick
Dodge feels unfair in PvP	Balance with low percentages (10-20%)
Accessories as optional dep	`compileOnly` + runtime class check
Armor textures	Recolor vanilla iron/netherite with unique effects
Progression too fast	Require previous tier as base (sequential)

What's Next

We have swords, spells, machines, and armor. But what about NPCs that sell you things? In the final post of this series, we'll build Cloud Caravan Merchants — wandering traders with donkeys, loot chests, and an Elite progression system. Because every cloud platform needs a marketplace.

Connect with me:

I'm Carlos Cortez, this is Breaking the Cloud, and today we armored up with cloud certifications. See you in the next one! 🛡️

I Built a Magic System Where Every Spell is a Cloud Operation

Carlos Cortez 🇵🇪 [AWS Hero] — Sat, 01 Aug 2026 15:13:45 +0000

What does casting a spell have in common with calling an API? More than you'd think. Both consume resources (mana/runes vs. API credits), both have cooldowns (cast time vs. rate limits), both have tiers (spell levels vs. service tiers), and both produce effects at a distance.

So we built Cloud Spellbooks — 9 tomes, 28 spells, 5 rune types, and a full upgrade path. Every spell is named after a cloud operation, and every mechanic maps to how cloud services actually work.

🎮 The learning angle: When you cast "Auto Scale" and it does AoE damage that scales with targets, you're experiencing what Auto Scaling does. When "Region Failover" fully heals you with a 5-minute cooldown, you understand why failover is a last resort with a long recovery time. The game mechanics ARE the cloud concepts.

The Architecture: Spell Engine Integration

We didn't build a spell system from scratch. We integrated with Spell Engine — a Fabric mod that provides data-driven spell registration, the Binding Table mechanic, and compatibility with other magic mods.

Our spells are defined as JSON:

{
  "school": "arcane",
  "range": 6.0,
  "cast": { "duration": 0, "channel_ticks": 0 },
  "cost": { "item_id": "cloudswords:packet_rune", "count": 1 },
  "cooldown": 3.0,
  "impact": [
    { "action": { "type": "spell_engine:damage", "damage": { "spell_power_coefficient": 0.5 } } },
    { "action": { "type": "spell_engine:status_effect", "status_effect": { "effect_id": "minecraft:glowing", "duration": 100 } } }
  ]
}

This is Infrastructure as Code for magic. Define your spell's behavior declaratively, and the engine handles execution.

9 Tomes, 3 Tiers — Like Service Tiers

Tier	Tomes	Rune Cost	Spell Power	Cloud Analogy
T1	Hacker's Notebook, Codex of Uptime	Packet Rune	+2	Free tier — basic operations
T2	Pipeline, SRE Codex, Cloud Captain, DevOps	Protocol Rune	+4	Standard tier — production workloads
T3	Serverless Bible, Grimoire of Architect, Zero Downtime	Quantum Rune	+7	Enterprise tier — mission-critical

Each tier requires a more expensive rune to cast. Just like cloud services — basic operations are cheap, enterprise features cost more.

28 Spells — Cloud Operations as Game Mechanics

Here are some highlights showing how cloud concepts become spells:

Spell	Tome	Effect	Cloud Concept
Ping	Hacker's Notebook	Damage + Glowing	Network ping — reveals target
DDoS Flood	Hacker's Notebook	Triple rapid hit	Distributed denial — overwhelming force
Firewall Burst	Codex of Uptime	Resistance + Fire Res	Firewall — blocks incoming
Auto Scale	Tome of Pipeline	AoE damage radius 6	Auto Scaling — handles multiple targets
Incident Response	SRE Codex	Regen III + Absorption	Incident response — emergency healing
Load Balance	Cloud Captain	Strength + Speed to allies	Load balancer — distributes power
CI/CD Sprint	DevOps Manifesto	Speed III + Haste II	CI/CD — everything faster
Cloud Nuke	Serverless Bible	Massive AoE	Nuclear option — destroy everything
Quantum Tunnel	Grimoire of Architect	Teleport 20 blocks	Quantum networking — instant transfer
Region Failover	Zero Downtime	Full heal + Res IV	Failover — complete recovery, long CD

The cooldowns tell a story too: "Ping" is 3 seconds (quick diagnostic), "Region Failover" is 5 minutes (disaster recovery is expensive and slow).

Rune System — API Credits

Runes are consumed when you cast spells. They're your API credits:

Rune	Tier	Crafting	Analogy
Packet Rune	T1	Cloud Shard + Redstone + Lapis	Basic API call
Protocol Rune	T2	Packet Rune + Blazing Shard + Gold	Standard API call
Quantum Rune	T3	Protocol Rune + Quantum Shard + Ender Pearl	Enterprise API call

Higher-tier spells consume more expensive runes. You can't spam "Cloud Nuke" because Quantum Runes are expensive to craft — just like you can't spam expensive API calls without budget consequences.

Upgrade Paths — Service Evolution

Hacker's Notebook → Tome of the Pipeline → The Serverless Bible
Codex of Uptime → The SRE Codex → Grimoire of Zero Downtime
                → Tome of the Cloud Captain → Grimoire of the Architect
                → DevOps Manifesto (independent)

Upgrades use the Smithing Table: Tome + Compiled/Deployed Rune + vanilla item. The path from T1 to T3 mirrors how cloud services evolve — you start with basic tools and upgrade to enterprise solutions.

Empowered Tomes — Premium Tier

Each tome has an "Empowered" variant (+2 extra spell power) crafted with rare materials:

T1 Empowered: + Amethyst Shard
T2 Empowered: + Blaze Rod
T3 Empowered: + Nether Star

It's like going from standard to reserved instances — same service, better performance, higher upfront cost.

Right-Click Casting — The UX

public abstract class SpellTome extends Item {
    @Override
    public TypedActionResult<ItemStack> use(World world, PlayerEntity player, Hand hand) {
        // 1. Check rune in inventory
        if (!hasRune(player, getRequiredRune())) {
            player.sendMessage(Text.literal("§c✗ Missing " + getRuneName()));
            return TypedActionResult.fail(stack);
        }
        // 2. Check cooldown
        if (isOnCooldown(player)) return TypedActionResult.fail(stack);
        // 3. Consume rune
        consumeRune(player, getRequiredRune());
        // 4. Cast spell
        castSpell(world, player);
        // 5. Apply cooldown
        applyCooldown(player);
        return TypedActionResult.success(stack);
    }
}

Simple flow: verify resources → consume → execute → cooldown. It's the same pattern as any API call: authenticate → charge → process → rate limit.

Lessons Learned

Challenge	Solution
28 spells is a lot of JSON	Data-driven approach — one template, many configs
Balancing cooldowns	Map to real-world operation frequency
Rune economy	Make T1 cheap, T3 expensive — natural gating
Spell Engine compatibility	Use `is_proxy: true` for Binding Table
Visual identity per tome	Isometric textures with unique colors + symbols

What's Next

Magic is great, but it needs infrastructure to support it. In the next post, we'll build a multiblock Data Melter with BFS energy transfer through cables — because even in a fantasy world, you need power grids.

Connect with me:

I'm Carlos Cortez, this is Breaking the Cloud, and today we cast cloud operations as spells. See you in the next one! ✨

Building a Power Grid Inside Minecraft with BFS Algorithms

Carlos Cortez 🇵🇪 [AWS Hero] — Mon, 13 Jul 2026 22:44:32 +0000

Every cloud service needs power. AWS has data centers, Azure has regions, GCP has zones — and they're all connected by networks that distribute energy and data. So we built the same thing in Minecraft: a multiblock Data Melter, energy storage blocks, cable networks with BFS traversal, and wireless energy transmission.

This is the post where our cloud-themed mod becomes a proper tech mod.

🎮 The learning angle: The BFS energy transfer algorithm is the same algorithm used in real network routing. The multiblock structure teaches you about distributed systems — one controller, multiple workers, shared resources. And energy storage tiers mirror how cloud providers offer different storage classes (S3 Standard, Infrequent Access, Glacier).

The Data Melter — A 3x3x3 Multiblock

The Data Melter is a 27-block structure:

22 Data Melter Casings — the shell
1 Bandwidth Block — the center (animated digital fluid)
1 Controller — the top (where you interact)

Click any casing to open the GUI (it searches for the controller in radius 3). This is how distributed systems work — any node can be your entry point, but they all route to the controller.

// Find controller from any casing click
private DataMelterControllerBlockEntity findController(World world, BlockPos pos) {
    for (BlockPos check : BlockPos.iterateOutwards(pos, 3, 3, 3)) {
        if (world.getBlockEntity(check) instanceof DataMelterControllerBlockEntity ctrl) {
            return ctrl;
        }
    }
    return null;
}

Recipes: Shard Infusion

The Data Melter processes materials that regular crafting can't:

Cloud Shard → Data-Infused Cloud Shard
Blazing Shard → Data-Infused Blazing Shard
Quantum Shard → Data-Infused Quantum Shard
Plus bonus yield recipes for vanilla materials

BFS Energy Transfer — Network Routing

The most technically interesting feature: energy flows through cables using Breadth-First Search.

When the Data Melter (or Cloud Generator) produces energy, it doesn't just teleport to consumers. It travels through cables — Internet Cables and Fiber Optic Cables — searching for connected machines that need power.

private void distributeEnergy(ServerWorld world, BlockPos source, int energy) {
    Queue<BlockPos> queue = new LinkedList<>();
    Set<BlockPos> visited = new HashSet<>();
    List<BlockPos> consumers = new ArrayList<>();

    queue.add(source);
    visited.add(source);

    // BFS through cable network (max 256 blocks)
    while (!queue.isEmpty() && visited.size() < 256) {
        BlockPos current = queue.poll();
        for (Direction dir : Direction.values()) {
            BlockPos neighbor = current.offset(dir);
            if (visited.contains(neighbor)) continue;
            visited.add(neighbor);

            if (isCable(world, neighbor)) {
                queue.add(neighbor); // Continue searching
            } else if (isConsumer(world, neighbor)) {
                consumers.add(neighbor); // Found a machine
            }
        }
    }

    // Distribute evenly among consumers
    int perConsumer = energy / consumers.size();
    for (BlockPos consumer : consumers) {
        addEnergy(world, consumer, perConsumer);
    }
}

This is literally how network packets find their destination. BFS explores all paths equally, finds all connected consumers, and distributes load evenly — like a load balancer.

Energy Storage — Tiered Like Cloud Storage

Block	Capacity	Cloud Analogy
Network Buffer	10 GB	S3 Standard — fast, limited
Bandwidth Accumulator	50 GB	S3 Infrequent Access — more capacity
Cloud Gateway Core	200 GB	S3 Glacier — massive storage

Each has a GUI with a color-coded energy bar (green → orange → red as it fills) and one storage slot.

Wireless Energy — Like WiFi

For when cables aren't practical:

Energy Transmitter — broadcasts energy in radius 8
Energy Receiver — picks up wireless energy

It's WiFi for power. Limited range, no cables needed, but less efficient than wired connections. Just like real networking — wired is faster and more reliable, wireless is convenient.

Cloud Generator — The Power Plant

The Cloud Generator converts fuel into energy:

Cloud Shard: 500 MB
Redstone: 50 MB
Coal/Charcoal: 25 MB

It produces energy at 100 MB/s with 45 GB internal storage. Connected via cables to your machines.

Cloud Engineer Lens — The Monitoring Dashboard

A wearable item (helmet, trinket, or held) that shows an HUD overlay with:

Energy levels of nearby machines
Connection status of cables
Production/consumption rates

It's CloudWatch for your Minecraft power grid. Observable by default.

Trinkets Integration

public static boolean isWearing(PlayerEntity player) {
    // Priority: Trinket slot → Helmet → Offhand → Mainhand
    if (FabricLoader.getInstance().isModLoaded("trinkets")) {
        if (TrinketsCompat.isInTrinketSlot(player)) return true;
    }
    // Fallback checks...
}

Optional mod support with runtime detection — the mod works with or without Trinkets installed.

Vertical Cable Fix — A War Story

Here's a debugging story. Internet Cables connect horizontally fine, but vertical connections (up/down) were broken. The multipart blockstate model uses X-axis rotations for vertical variants, but 1.20.1's renderer doesn't handle X-axis rotations correctly for multipart models.

The fix: create dedicated _up and _down models with pre-positioned geometry instead of relying on rotation. Sometimes the elegant solution doesn't work and you need the pragmatic one.

Lessons Learned

Challenge	Solution
BFS can be expensive	Cap at 256 blocks, run once per tick
Multiblock detection	Search from any block, find controller
Vertical cables broken	Dedicated models instead of rotation
Energy sync to client	`toUpdatePacket` + `toInitialChunkDataNbt`
GUI without texture files	Programmatic rendering (DrawContext)

What's Next

We have power, machines, and networks. Now we need protection. In the next post: 4 armor sets with set bonuses, a dodge system, and integration with the Accessories mod. Because in the cloud, security is a shared responsibility.

Connect with me:

I'm Carlos Cortez, this is Breaking the Cloud, and today we built a power grid with graph algorithms. See you in the next one! ⚡

Backporting a Minecraft Mod Across Versions — And Building a Cloud Economy with Villagers

Carlos Cortez 🇵🇪 [AWS Hero] — Mon, 13 Jul 2026 22:13:55 +0000

We had a problem. The mod was called "AWS Swords," built for Minecraft 1.21.1, and it worked great in isolation. But nobody plays Minecraft in isolation. The biggest modpacks — Prominence II, All The Mods, Create-based packs — they're all on 1.20.1. And using "AWS" in the name raised trademark questions.

So we did what any good cloud architect does: we refactored, rebranded, and migrated. Welcome to Cloud Swords Mod — now running on 1.20.1, compatible with 650+ mods, and featuring a full villager-based cloud economy.

🎮 The learning angle: Backporting a mod is like migrating between cloud provider API versions. The concepts are the same, but every method signature is slightly different. And building a villager economy teaches you about service catalogs, tiered pricing, and progression systems — all concepts from real cloud platforms.

The Rebrand: AWS Swords → Cloud Swords

First, the easy part. We renamed everything:

awsswords → cloudswords
"AWS Swords" → "Cloud Swords Mod"
Service-specific names stayed (Lambda, S3, EC2) but as game lore, not branding

Why? Because we want to publish on CurseForge/Modrinth, and using a company's trademark in a mod name is risky. The swords are inspired by cloud services — they don't claim to be official AWS products.

The Backport: 1.21.1 → 1.20.1

This is where it gets technical. Minecraft 1.21.1 and 1.20.1 look similar from a player's perspective, but the modding API changed significantly. Here's every difference we hit:

1.21.1	1.20.1	Impact
`Identifier.of("ns", "path")`	`new Identifier("ns", "path")`	Every single identifier
`DataComponentTypes.CUSTOM_DATA`	Direct NBT on ItemStack	Complete rewrite of S3 sword storage
`fabric-loot-api-v3`	`fabric-loot-api-v2`	Different lambda signatures
`LootTableEvents.MODIFY` 4 params	5 params, different types	Loot injection rewrite
`Equipment` interface	Doesn't exist	Remove entirely
`BlockEntityType.Builder.build()`	`build(null)`	Every block entity
Java 21	Java 17	Language features
EMI `1.1.16+1.21.1`	`1.1.22+1.20.1`	Different API surface

The lesson? API migrations are never "just find and replace." Every change has cascading effects. The DataComponentTypes change alone required rewriting how the S3 sword stores items — from the new component system back to raw NBT compounds.

// 1.21.1 — new component system
stack.getOrDefault(DataComponentTypes.CUSTOM_DATA, NbtComponent.DEFAULT)
    .copyNbt().getList("stored_items", NbtElement.COMPOUND_TYPE);

// 1.20.1 — direct NBT
stack.getOrCreateNbt().getList("stored_items", NbtElement.COMPOUND_TYPE);

Same concept, completely different API. Sound familiar? It's like migrating from AWS SDK v2 to v3.

7 Cloud Villager Professions

The biggest new feature: a full economy system powered by villagers. Each profession represents a cloud role:

Profession	Workstation	Specialty
Software Developer	Cloud Deployer	Sword cores, Lambda chips
Cloud Architect	Cloud Console	Network components, VPC frames
Security Engineer	Security Terminal	Firewall modules, encryption
AI Engineer	AI Workbench	Processors, auto-scaling modules
DevOps Engineer	Cloud Generator	Energy, cables, automation
SysAdmin	Server Rack	Server components, overclocking
Data Engineer	Cloud Refinery	Dusts, data processing

Each has 5 trade levels (Novice → Master), and Master-level trades accept Cloud Credits — a custom currency that unlocks premium items.

Master-Level Exclusive Items

These are items you can ONLY get from Master villagers with Cloud Credits:

Item	Effect	Cloud Concept
Serverless Function	Teleport to last death location	Lambda — runs on demand
Multi-Region Backup	Restore inventory on death	S3 Cross-Region Replication
Zero Downtime Shield	Prevent death once	High Availability
Training Dataset	+30 XP levels instantly	ML Training
Infinite Bandwidth Token	Haste III + Speed II for 5 min	Network throughput
Data Pipeline	Duplicate all dusts in inventory	ETL processing

Plus vanilla rare items (Totem, Elytra, Nether Star) for Cloud Credits — because in the cloud, money solves problems.

Cloud Office — Village Structure

We injected a custom structure into vanilla villages: the Cloud Office. It's a 7x7x7 building with:

3 workstations (Security Terminal, AI Workbench, Server Rack)
Copper accents for visual distinction
Spawns in ~50% of villages

Technically, this uses Fabric's ServerLifecycleEvents.SERVER_STARTING to inject a jigsaw element into the village structure pool before world generation. An access widener exposes the StructurePool.elements field.

Ore Generation Across Dimensions

Ore	Dimension	Y Range	Vein Size	Per Chunk
Cloud Ore	Overworld	0-64	6	4
Nether Cloud Ore	Nether	30-90	4	3
End Cloud Ore	End	0-80	3	2

The progression mirrors cloud regions: you start local (Overworld), expand to a secondary region (Nether), and eventually go global (End).

Prominence II Compatibility — 650+ Mods

The ultimate test: does it work in a modpack with 650 mods? Yes.

No item/block ID conflicts
Energy system coexists with Create, Tech Reborn, Mekanism
Villager professions work alongside VillagersPlus
Ore generation works with Regions Unexplored biomes
EMI shows all recipes correctly
Loot injection works in all structure types

The key was using Fabric's registry system correctly and not hardcoding anything. Namespaced identifiers, proper tags, and optional dependencies.

Lessons Learned

Challenge	Solution
API differences between versions	Document every change, migrate systematically
Trademark concerns	Rebrand early, use "inspired by" language
650-mod compatibility	Use registries, avoid hardcoding, test early
Village structure injection	Access widener + jigsaw pool manipulation
Economy balance	Cloud Credits as gating currency for endgame

What's Next

The foundation is solid: 7 swords, machines, villagers, economy, ores, and full modpack compatibility. But we're just getting started. In the next post, we'll add a complete magic system — 28 spells across 9 spellbooks, each themed around cloud operations. Because in the cloud, automation is magic.

Full source on GitHub.

Connect with me:

I'm Carlos Cortez, this is Breaking the Cloud, and today we migrated to a new region. See you in the next one! 🌍

I Built a CI/CD Pipeline and an AWS Console Inside Minecraft

Carlos Cortez 🇵🇪 [AWS Hero] — Mon, 13 Jul 2026 22:12:51 +0000

What if your Minecraft crafting system worked like cloud infrastructure? Not just "put items in a grid and get a result" — but actual deployment pipelines with processing time, consoles that randomize configurations like spinning up instances, and a tiered progression that mirrors going from dev to staging to production.

In Part 2, we made the swords look good. Now we make the infrastructure that produces them feel like real cloud operations.

🎮 The learning angle: Every machine in this mod maps to a real cloud concept. The Cloud Deployer IS a CI/CD pipeline — it takes source code (Inactive Core) and produces a deployable artifact (Active Core) over time. The Cloud Console IS the AWS Console — you configure your resources (swords) with randomized parameters within tier-specific ranges.

The Cloud Deployer — Your CI/CD Pipeline

In the real world, you don't just write code and instantly have a running service. You push to a repo, a pipeline picks it up, builds it, tests it, and deploys it. That takes time.

The Cloud Deployer works the same way:

Input: Inactive Sword Core (your "source code")
Process: 8-15 seconds of "deployment" (progress bar)
Output: Active Sword Core (your "deployed artifact")

Two tiers of deployer:

Cloud Deployer (Low) — 15 seconds, produces Basic Core. Like a free-tier CI pipeline.
Cloud Deployer (Medium) — 8 seconds, produces Advanced Core. Like upgrading to a faster build runner.

The Basic Core only works for Tier 1 swords (CloudWatch). The Advanced Core unlocks everything else. Just like in real cloud — your free tier has limitations, but upgrading your infrastructure unlocks more powerful services.

The Code: A BlockEntity with a Timer

public class CloudDeployerBlockEntity extends BlockEntity implements TickableBlockEntity {
    private int progress = 0;
    private final int maxProgress; // 15s or 8s depending on tier

    @Override
    public void tick() {
        if (hasInput() && !hasOutput()) {
            progress++;
            if (progress >= maxProgress) {
                craftOutput(); // Inactive Core → Active Core
                progress = 0;
            }
        }
    }
}

Simple, but the game feel is powerful. You place your core, watch the progress bar fill, and get your artifact. It feels like deploying.

The Cloud Console — Randomized Configuration

Here's where it gets interesting. The Cloud Console is a block where you place a sword and it randomizes its stats within tier-specific ranges.

Think of it like the AWS Console where you configure an EC2 instance:

You choose your instance type (tier)
You get specific specs within that family's range
Higher tiers give better ranges

Console Tier	Damage Range	Speed Range	Cooldown Range
Low	-3 to +5	-2 to +1	-2s to 0s
Medium	-3 to +10	-2 to +2	-4s to 0s
Creative	-3 to +10	-2 to +3	-5s to 0s

Every time you use the console, you get a different roll. Maybe your Lambda sword gets +8 damage but -1 speed. Maybe next time it's +3 damage but +2 speed. It's like choosing between compute-optimized and memory-optimized instances — tradeoffs everywhere.

public enum ConsoleTier {
    LOW(-3, 5, -2, 1, -2, 0),
    MEDIUM(-3, 10, -2, 2, -4, 0),
    CREATIVE(-3, 10, -2, 3, -5, 0);

    // Random roll within range
    public int rollDamage(Random random) {
        return random.nextBetween(minDmg, maxDmg);
    }
}

Deprecated Swords — Legacy Migration

One of my favorite design decisions: Deprecated Swords.

These are weaker versions of each sword that drop as loot in dungeons and structures. They have iron-level stats, no abilities, and a tooltip that says: "A legacy service awaiting migration..."

Sound familiar? It's the cloud migration story. You find a legacy system (deprecated sword in a dungeon), and you can either:

Use it as-is — works, but limited
Migrate it — take it to the Smithing Table with an Active Core and upgrade it to the full version

The migration recipe is cheaper than crafting from scratch, just like in real cloud — migrating existing workloads is often more cost-effective than rebuilding.

The Progression: Dev → Staging → Production

The full player journey maps perfectly to cloud maturity:

EARLY GAME (Overworld) = Development Environment
  Mine Cloud Ore → Basic materials
  → Cloud Console (Low): limited configuration
  → Cloud Deployer (Low): slow deployments, basic artifacts
  → Only Sword of CloudWatch accessible (monitoring first!)

MID GAME (Nether) = Staging Environment  
  Mine Nether Cloud Ore → Advanced materials
  → Cloud Console (Medium): better configuration ranges
  → Cloud Deployer (Medium): faster deployments, advanced artifacts
  → All swords accessible (full service catalog)

Notice that CloudWatch is the first sword you can craft. In real AWS, observability should be the first thing you set up. The game teaches this naturally.

EMI Integration — Your Service Catalog

EMI (Extremely Mod Integration) is Minecraft's recipe viewer. We added a custom "Deploying" category that shows:

What goes in (Inactive Core)
What comes out (Active Core)
How long it takes (progress bar animation)

It's literally a service catalog. Players can browse all available "deployments" and see what infrastructure they need.

Smithing Table — Infrastructure as Code

The Smithing Table recipes follow a pattern:

Template: Active Sword Core (your deployment artifact)
Base: Service-specific component (Lambda Chip, Processor Core, etc.)
Addition: Cloud material (Blazing Shard, Network Cable, etc.)

This is Infrastructure as Code. You define your template (the core), specify your service (the base component), and add your configuration (the addition). The Smithing Table "applies" your configuration and produces the final resource.

Lessons Learned

Challenge	Solution
Randomized stats feel unfair	Show the ranges clearly in tooltip
Deployment time feels slow	Add progress bar + particles
Too many recipes to remember	EMI integration is essential
Tier progression unclear	Visual differentiation (colors per tier)
Legacy migration concept	Deprecated swords as loot → upgrade path

What's Next

We have swords, textures, machines, and progression. But we're still on Minecraft 1.21.1 with a mod called "AWS Swords." In the next post, we'll rebrand to Cloud Swords, add a full villager economy with 7 cloud professions, and backport everything to Minecraft 1.20.1 for compatibility with the biggest modpacks.

Because a mod isn't real until it works alongside 650 other mods.

Full source code on GitHub.

Connect with me:

I'm Carlos Cortez, this is Breaking the Cloud, and today we deployed swords to production. See you in the next one! 🚀

# I Built an MCP Server That Investigates Your AWS Bill

Carlos Cortez 🇵🇪 [AWS Hero] — Thu, 02 Jul 2026 16:52:18 +0000

I Built an MCP Server That Investigates Your AWS Bill

TL;DR: Install one MCP server → ask your AI "why did my bill spike?" → get a full investigation with resource IDs, dollar amounts, and CLI commands to fix it. Open source, 8 tools, works with Kiro, Claude Code, and Cursor.

The Problem Every Cloud Engineer Knows

Every month, the same ritual. You open AWS Cost Explorer, click through dashboards, squint at bar charts, try to figure out what changed. Then you open EC2 to check for idle instances. Then you check tags — half your resources don't have them. Then you wonder if someone left a NAT Gateway running in a sandbox account.

This process takes 30-60 minutes. And you do it every month. Sometimes every week when something spikes.

I've been building cloud architectures for years, and this friction never went away. The data is there — Cost Explorer has it, CloudWatch has it, the Tagging API has it. But the workflow of pulling it all together, correlating it, and turning it into actionable recommendations? That's still manual.

What if your AI assistant could do all of that in one question?

That's what I built.

The Solution: byaml-finops-mcp

pip install byaml-finops-mcp

It's an MCP server — a standard protocol that lets AI assistants call external tools. Once installed, your AI (Kiro, Claude Code, Cursor, or any MCP-compatible client) gains 8 new capabilities that connect directly to your AWS account:

Tool	What it does
`get_cost_by_service`	Monthly costs by service, current vs previous month
`get_cost_anomalies`	Detected cost spikes with root causes
`find_idle_resources`	EC2 with <5% CPU, unattached EBS volumes
`estimate_termination_savings`	Exact monthly savings from terminating instances
`get_savings_recommendations`	Rightsizing + Savings Plans
`detect_missing_tags`	Resources missing owner/env/costCenter tags
`generate_byaml_from_account`	Scan account → generate architecture-as-code
`validate_byaml`	Validate architecture files against schema + policies

The key insight: your AI doesn't just answer questions about costs — it investigates them. It pulls data from multiple AWS APIs, correlates findings, and gives you specific resource IDs with dollar amounts and CLI commands to fix issues.

Real Results From My Account

I didn't build this with mock data. I ran it against my own AWS account — the one running my side projects, APIs, and experiments. Here's what it found in seconds:

Missing Tags: 46/46 resources non-compliant

Resources scanned: 46
Violations: 46 (100% non-compliant)
Required tags missing: owner, env, costCenter

Every single resource — Lambda functions, S3 buckets, API Gateways, CloudFront distributions — was missing the required governance tags. But here's the interesting part: the tool identified a case mismatch issue. Many resources had Owner and Environment (capitalized) but the policy requires lowercase owner and env. A subtle bug that would take forever to find manually.

Architecture Discovery: 27 components auto-detected

The generate_byaml_from_account tool scanned 7 AWS services (Lambda, S3, DynamoDB, API Gateway, CloudFront, RDS, EC2) and generated a complete architecture file:

byamlVersion: '0.3'
architecture:
  id: aws-us-east-1
  environment: prod
  cloudProvider: aws
components:
- id: byaml-schema-api-api
  type: aws.lambda
  props:
    runtime: python3.11
    memory: 256
- id: byaml-schema-registry
  type: aws.s3
# ... 25 more components

27 components discovered, mapped to typed BYaML component types, and output as a valid .byaml file. This is architecture-as-code generated from your live infrastructure — not from Terraform or CloudFormation, but from what's actually running.

Validation: Zero errors on generated output

{
  "valid": true,
  "findings": [],
  "summary": {"errors": 0, "warnings": 0, "total": 0}
}

The generated file validates cleanly against the v0.3 JSON Schema, a component catalog (13 valid types), and FinOps policy rules. If you add a component with a type that doesn't exist in the catalog, you get a warning. If you're missing required fields, you get an error. All in real-time.

How It Works

The architecture is intentionally simple:

Your AI (Kiro / Claude Code / Cursor)
        ↓ MCP Protocol (stdio)
byaml-finops-mcp (runs on your machine)
        ↓ boto3 (AWS SDK)
Your AWS Account (read-only API calls)
        ↓
Cost Explorer, EC2, CloudWatch, Lambda, S3, RDS, DynamoDB, Tagging API

Three design principles:

Local-first — the server runs on your machine. No data leaves your environment. No SaaS dashboard, no third-party ingestion.
Read-only — it never modifies your AWS resources. Only Describe*, List*, and Get* API calls.
Zero config — it uses your existing AWS CLI profile. If aws sts get-caller-identity works, the MCP server works.

Install in 2 Minutes

1. Install

pip install byaml-finops-mcp

Or with uvx (no install needed):

uvx byaml-finops-mcp

2. Configure your MCP client

Kiro (.kiro/settings/mcp.json):

{
  "mcpServers": {
    "byaml-finops": {
      "command": "uvx",
      "args": ["byaml-finops-mcp"],
      "env": { "AWS_PROFILE": "your-profile" }
    }
  }
}

Claude Code (~/.claude/mcp.json) and Cursor use the same format.

3. Ask your AI

"Why did my AWS bill spike this month?"

"Find idle resources and tell me how much I'd save by terminating them"

"Which resources are missing governance tags?"

"Generate a BYaML architecture file from my account"

The AI will call the appropriate tools, correlate the results, and give you a structured investigation with actionable next steps.

The BYaML Connection

The last two tools (generate_byaml_from_account and validate_byaml) connect to BYaML — an architecture-as-code format I'm building at Brickstore AI. Think of it as "OpenAPI spec, but for cloud architectures."

The schema is fully public:

schema.byaml.org/v1/byaml/schema/version/0.3 — JSON Schema
schema.byaml.org/v1/byaml/schema/version/0.3/catalog — Valid component types
schema.byaml.org/v1/byaml/schema/version/0.3/policies — FinOps policy rules
schema.byaml.org/v1/byaml/schema/version/0.3/relationships — Valid connections between components

The MCP server fetches the schema from this public API and validates locally. No API key needed. There's also a VS Code extension (brickstore-ai.byaml) that gives you syntax highlighting, autocomplete, and real-time validation for .byaml files.

The Backstory

This project evolved from a series of experiments I did with NVIDIA NIM + FinOps agents. The original version used mock billing data and a single-agent architecture. Then I rebuilt it with sub-agents (cost analyst, resource auditor, deploy correlator). Then I realized: the real value isn't in the agent framework — it's in the tools.

MCP gave me the perfect distribution mechanism. Instead of building a full agent that only works with one LLM, I built tools that work with any AI that speaks MCP. The intelligence comes from the AI client; the data comes from the tools.

What's Next

Terraform Connector — generate BYaML from tfplan.json (biggest gap for enterprise)
Drift Engine — compare your .byaml file against what's actually running in AWS
Multi-account — cross-account cost analysis via AWS Organizations
Premium tier — historical trends, AI-powered root cause analysis, architecture intelligence via api.dev.brickstore.ai

Try It

pip install byaml-finops-mcp

GitHub: brickstore-ai/byaml-finops-mcp

VS Code Extension: brickstore-ai.byaml

About

Built by Carlos Cortez at Brickstore AI — AWS Community Hero, cloud architect, and builder of AI-native developer tools.

If you're building in this space or have feedback on the MCP server, reach out:

🐦 X: @ccortezb
💼 LinkedIn: Carlos Cortez
🏢 Brickstore AI: LinkedIn
🌐 Web: cortez.cloud
🧑‍💻 GitHub: ccortezb
📝 Dev.to: ccortezb
☁️ AWS Heroes: @breakinthecloud

Generating 3D Animated Sword Textures with Python and AI — For a Minecraft Mod About Cloud Computing

Carlos Cortez 🇵🇪 [AWS Hero] — Thu, 25 Jun 2026 15:07:59 +0000

I Generated 3D Animated Sword Textures with Python and AI — For a Minecraft Mod About Cloud Computing

In Part 1, we built 4 swords where each one represents an AWS service. Lambda summons ephemeral allies, S3 stores items, EC2 auto-scales damage. The abilities worked. The code compiled. But the swords looked... generic.

This is the story of how I went from flat placeholder textures to fully animated, UV-mapped 3D sword models — using Python scripts, Blockbench, and 7 iterations of a texture generator. And why the visual design of a game item can teach you more about cloud services than a whitepaper ever could.

🎮 The learning angle: When you design a texture for "Sword of Lambda," you have to think about what Lambda represents visually. Ephemeral? Use fire that fades. Event-driven? Add energy pulses. Serverless? Make it feel lightweight. This design process forces you to internalize service characteristics in a way that reading docs never does.

The Problem: Minecraft Textures Are Harder Than They Look

Minecraft items use 16x16 pixel textures. That's 256 pixels to communicate:

What service this sword represents
What tier it belongs to
That it's special (not just another iron sword)
Its elemental/thematic identity

And for animated textures, you need 8 frames in a vertical sprite sheet (16x128) plus a .mcmeta file that controls animation speed and interpolation.

For 3D models (which we used via Blockbench), you need UV-mapped textures where specific pixel regions map to specific faces of the 3D geometry. One wrong pixel and your blade has a handle texture on it.

The Solution: Python + PIL + Iteration

Instead of hand-painting each texture in a pixel editor, I wrote Python scripts that generate textures programmatically. Here's why:

Consistency — All swords share the same UV layout structure
Iteration speed — Change a color palette, regenerate all textures instantly
Animation — Generate 8 frames with programmatic color shifts per frame
Theming — Each service gets a color palette that maps to its identity

Version 1: Basic UV Layout

#!/usr/bin/env python3
"""Generate UV-mapped textures for AWS Swords 3D models."""
from PIL import Image

def sword_of_lambda():
    """16x16 UV texture for Sword of Lambda.
    UV layout:
      Row 0-3:   blade (orange/purple gradient with energy)
      Row 4-5:   blade edge highlight
      Row 6-7:   crossguard (gold/dark)
      Row 8:     gem (bright orange center)
      Row 9-11:  handle (dark purple with grip lines)
      Row 12-13: pommel (gold)
      Row 14-15: energy glow / emissive accents
    """
    img = Image.new("RGBA", (16, 16), (0, 0, 0, 0))
    px = img.load()

    # Lambda theme: orange fire + purple energy
    blade_dark = (180, 80, 20)
    blade_mid = (230, 120, 30)
    purple_energy = (160, 50, 200)
    gem_core = (255, 160, 0)
    gold = (220, 180, 50)
    # ... fill rows with themed colors

This first version was functional but flat. No animation, no depth, no personality.

Version 6: Animated Frames with Energy Pulses

By version 6, the generator produced 8-frame sprite sheets with:

Frame-by-frame color cycling for energy effects
Gem pulsing (brightness oscillates across frames)
Edge highlights that shift position per frame
Enchant glint compatibility (transparent pixels in the right spots)

The .mcmeta file that controls animation:

{
  "animation": {
    "interpolate": true,
    "frametime": 4
  }
}

interpolate: true makes Minecraft blend between frames smoothly. frametime: 4 means each frame lasts 4 ticks (0.2 seconds), giving us a ~1.6 second full animation cycle.

Color Palettes as Cloud Identity

Here's where the cloud learning happens. Each sword needed a color palette that communicates the service:

Sword	Primary	Secondary	Why
Lambda	Orange/fire	Purple energy	Lambda's icon is orange. Ephemeral = fire that burns out
S3	Green/teal	Teal gems	S3's icon is green. Storage = solid, stable colors
EC2	Blue/cyan	Orange sparks	EC2's icon is orange on blue. Electric = scaling power
Greatsword	Purple/pink	Holy fire	Lambda variant but "holy" — massive invocations

When you play the mod, you can identify each sword at a glance. The visual language maps directly to the service identity. This is the same principle behind good cloud architecture diagrams — color coding services by category.

Blockbench: From Flat to 3D

Minecraft's default item rendering is a flat sprite held at an angle. But with Blockbench, you can create actual 3D geometry:

Blade: elongated cuboid with tapered edges
Crossguard: horizontal bar with decorative ends
Handle: wrapped cylinder with grip texture
Gem: small centered cube with emissive-looking colors

The model exports as a JSON file that Minecraft loads:

{
  "parent": "item/handheld",
  "textures": {
    "layer0": "awsswords:item/sword_of_lambda"
  },
  "display": {
    "thirdperson_righthand": {
      "rotation": [0, -90, 55],
      "translation": [0, 4.0, 0.5],
      "scale": [0.85, 0.85, 0.85]
    }
  }
}

7 Iterations — What Changed Each Time

Version	What Changed	Why
v1	Basic UV layout, flat colors	Proof of concept
v2	Added gradients within rows	Depth perception
v3	Gem highlights, edge glow	Visual focal point
v4	8-frame animation support	Movement = life
v5	Per-service color palettes	Identity at a glance
v6	Energy pulse patterns per frame	Unique animation per sword
v7	Enchant glint layer compatibility	Works with vanilla enchanting

Each iteration was a conversation: "This looks too flat" → add gradients. "I can't tell Lambda from EC2" → differentiate palettes. "The animation feels static" → add per-frame energy shifts.

The Cloud Computing Connection

Here's what surprised me: designing game visuals for cloud services forces you to understand them deeply.

To make Lambda's texture "feel ephemeral," I had to think about what ephemeral looks like. Fire that fades. Energy that pulses and disappears. Particles that spawn and despawn.

To make S3's texture "feel like storage," I needed stability. Solid green. Consistent glow. A gem that represents the "object" being stored.

To make EC2's texture "feel like scaling," I used electric sparks that multiply. More energy = more power. The visual metaphor IS the service behavior.

This is why game-based learning works. You're not memorizing — you're designing. And design requires understanding.

Lessons Learned

Challenge	Solution
UV mapping is confusing	Document the layout (which row = which part)
16x16 is tiny	Every pixel matters — use contrast aggressively
Animation needs subtlety	`interpolate: true` saves you from choppy frames
Color blindness	Don't rely on color alone — use shape/pattern too
7 script versions	Each iteration taught something new about PIL

What's Next

The textures are done, but the mod needs more than pretty swords. In the next post, we'll build the Cloud Console — a block that randomizes sword stats like choosing an EC2 instance type. And the Cloud Deployer — a time-based crafting machine that works like a CI/CD pipeline.

Because in cloud computing, you don't just have tools — you have infrastructure. And infrastructure needs machines.

**The full texture generation scripts (all 7 versions) are:

Connect with me:

I'm Carlos Cortez, this is Breaking the Cloud, and today we painted the cloud in 256 pixels. See you in the next one!

Amazon Q Developer CLI ahora es Kiro CLI — ¿Qué cambió y por qué importa?

Carlos Cortez 🇵🇪 [AWS Hero] — Thu, 21 May 2026 01:03:15 +0000

Amazon Q Developer CLI ahora es Kiro CLI — ¿Qué cambió y por qué importa?

Si llevas un tiempo en el ecosistema AWS y usas herramientas de desarrollo con IA, probablemente ya notaste el cambio: Amazon Q Developer CLI ya no existe como tal. Ahora se llama Kiro CLI. Y no, no es solo un rebrand de nombre — es un cambio de filosofía completo.

Vamos a explorar qué pasó, qué cambió realmente, y por qué creo que esto importa más de lo que parece.

Un poco de contexto: ¿qué era Amazon Q Developer CLI?

Amazon Q Developer era el asistente de IA de AWS para desarrolladores. Tenía una versión en el IDE (VS Code, JetBrains), una versión en la consola de AWS, y también una CLI que te permitía interactuar con tu entorno desde la terminal usando lenguaje natural.

La idea era buena: preguntarle a un agente directamente desde tu terminal cosas como "¿qué instancias EC2 tengo corriendo en us-east-1?" o "genera un script para limpiar buckets S3 sin versioning". Útil, pero limitado en su enfoque — era básicamente un chatbot en tu terminal.

Entonces, ¿qué es Kiro?

Kiro es un IDE agéntico construido sobre VS Code, lanzado por AWS. Pero lo que muchos no saben es que también tiene una CLI — Kiro CLI — que reemplaza directamente a Amazon Q Developer CLI.

Lo interesante es que Kiro no es solo "Q con otro nombre". El cambio refleja una evolución real en cómo AWS piensa las herramientas de desarrollo:

Amazon Q Developer CLI	Kiro CLI
Asistente conversacional en terminal	Agente con contexto del proyecto
Respuestas puntuales	Spec-driven + MCP-driven + Steering-driven
Foco en queries rápidas	Foco en workflows completos
Sin memoria de proyecto	Entiende tu arquitectura y convenciones
Comando: `q chat`	Comando: `kiro-cli chat`

La idea aquí es que Kiro no solo responde preguntas — razona sobre tu proyecto, lee tus steering files, se conecta a herramientas externas vía MCP, y actúa en consecuencia.

El cambio más importante: de "asistente" a "agente"

En la práctica esto significa que Kiro CLI opera con capacidades que Amazon Q Developer CLI nunca tuvo de forma nativa:

Core Features de Kiro CLI

Interactive Chat — Conversaciones en lenguaje natural directamente en tu terminal con kiro-cli chat. Puede leer y escribir archivos, ejecutar comandos, y razonar sobre tu código.
Custom Agents — Puedes crear y desplegar agentes especializados para tus workflows específicos. No estás limitado al agente genérico.
MCP Integration — Conecta herramientas y fuentes de datos externas a través del Model Context Protocol. Esto es enorme — puedes conectar Kiro CLI a servidores MCP de CloudWatch, MSK, OpenSearch, Okta, y muchos más.
Smart Hooks — Automatiza workflows con hooks inteligentes que se ejecutan antes o después de comandos específicos.
Agent Steering — Guía al agente con las mejores prácticas y preferencias de tu equipo usando steering files. Esto es lo que hace que Kiro entienda tu contexto, no solo el contexto genérico.
Auto Complete — Sugerencias inteligentes de comandos con contexto mientras escribes en la terminal.

# Antes con Amazon Q Developer CLI
q chat "lista mis funciones Lambda en us-east-1"

# Ahora con Kiro CLI — con contexto de proyecto y MCP
kiro-cli chat
> revisa las funciones Lambda del proyecto y sugiere optimizaciones basándote en las métricas de CloudWatch

La diferencia no es solo sintáctica. Kiro sabe qué proyecto es, tiene acceso a tus herramientas vía MCP, y puede actuar sobre eso.

¿Cómo instalar Kiro CLI hoy?

La instalación es directa:

# macOS / Linux
curl -fsSL https://cli.kiro.dev/install | bash

# Windows (PowerShell)
irm 'https://cli.kiro.dev/install.ps1' | iex

# Verificar instalación
kiro-cli --version

Una vez instalado, empezar es así de simple:

# Navega a tu proyecto
cd mi-proyecto

# Inicia Kiro CLI
kiro-cli

# O directamente al chat
kiro-cli chat

Otros comandos útiles

# Traducir lenguaje natural a comandos bash
kiro-cli translate "muestra los últimos 10 logs de mi función Lambda"

# Habilitar sugerencias inline (requiere zsh)
kiro-cli inline enable

# Deshabilitar sugerencias inline
kiro-cli inline disable

Lo interesante es que kiro-cli translate convierte tu instrucción en el comando bash correspondiente sin ejecutarlo — tú decides si lo corres o no. Perfecto para aprender comandos complejos de AWS CLI.

Kiro CLI en CloudShell

Si no quieres instalar nada localmente, Kiro CLI ya viene disponible en AWS CloudShell. Solo abre CloudShell y ejecuta kiro-cli.

Las sugerencias inline en CloudShell requieren Z shell:

# Cambiar a zsh en CloudShell
zsh

# Las sugerencias inline se habilitan automáticamente
# Para deshabilitarlas:
kiro-cli inline disable

Agent Steering: dale contexto persistente a Kiro CLI

Esta es una de las features más importantes de Kiro CLI y la que marca la diferencia real con lo que teníamos en Amazon Q Developer CLI. Los steering files son archivos markdown que le dan a Kiro conocimiento persistente sobre tu proyecto, tu stack, y las convenciones de tu equipo.

La idea aquí es simple: en vez de re-explicar tu proyecto cada vez que abres una sesión, escribes un steering file una vez y Kiro lo lee automáticamente en cada interacción.

¿Qué poner en un steering file?

Un steering file es un .md que vive en tu proyecto (típicamente en .kiro/steering/) y puede contener:

Stack tecnológico — qué lenguajes, frameworks y servicios usas
Convenciones del equipo — naming conventions, patrones de diseño, estructura de carpetas
Contexto de infraestructura — nombres de instancias, ubicación de logs, usuarios del sistema
Requisitos de compliance — estándares de seguridad, accesibilidad, auditoría
Reglas de negocio — lógica específica de tu dominio que el agente debe respetar

Ejemplo real: steering file para un proyecto serverless

# Project Context

## Stack
- Runtime: Python 3.12 on AWS Lambda
- API: Amazon API Gateway (REST)
- Database: Amazon DynamoDB (single-table design)
- IaC: AWS CDK (Python)

## Conventions
- All Lambda handlers go in `src/handlers/`
- Business logic goes in `src/services/` — never in handlers
- Use structured logging with aws-lambda-powertools
- DynamoDB access patterns use PK/SK with GSI1

## Security
- All API endpoints require IAM authorization
- No hardcoded credentials — use environment variables from Secrets Manager
- Input validation on every handler entry point

Lo que hace esto particularmente poderoso es que Kiro CLI usa este contexto en cada interacción. Si le pides que genere un nuevo endpoint, va a seguir tus convenciones automáticamente — handlers en src/handlers/, lógica en src/services/, con powertools y validación de input. Sin que tengas que repetirlo.

Steering files en la práctica

El blog de AWS sobre Oracle EBS con Kiro CLI muestra un caso real: usan steering files para darle a Kiro el conocimiento de su entorno Oracle — patrones de nombres de instancias, usuarios del OS, ubicación de logs y scripts. Así, cuando preguntan "¿está sano el concurrent manager?", Kiro ya sabe dónde buscar sin que se lo expliquen cada vez.

Para equipos, esto es oro. Un nuevo developer se une, clona el repo, y Kiro CLI ya tiene todo el contexto del proyecto en los steering files. La curva de onboarding se reduce dramáticamente.

Spec-Driven Development: ¿disponible en Kiro CLI?

Acá hay que ser claros porque es una pregunta que muchos se hacen. Spec-Driven Development es una feature del Kiro IDE, no de Kiro CLI.

Según la documentación oficial de Kiro, las Specs viven bajo la sección de documentación del IDE (/docs/specs/), y no aparecen en la sidebar de la CLI. La CLI tiene: Chat, Custom Agents, MCP, Hooks, Steering, Autocomplete, y Headless — pero no Specs.

¿Qué es Spec-Driven Development?

Para los que no lo conocen, es el workflow estrella de Kiro IDE. Funciona así:

Le describes tu idea al agente en lenguaje natural
Kiro genera requirements estructurados (en formato EARS)
Kiro crea un design document con arquitectura, modelos de datos, APIs
Kiro produce un plan de implementación con tareas concretas y ordenadas
Kiro ejecuta cada tarea — escribe código, tests, documentación

Cada spec genera tres archivos clave: requirements.md, design.md, y tasks.md. Hay dos tipos de specs: Feature Specs (para funcionalidades nuevas) y Bugfix Specs (para diagnosticar y corregir bugs).

¿Por qué no está en la CLI?

Mi lectura es que Spec-Driven Development requiere una experiencia visual que la terminal no puede ofrecer fácilmente — la navegación entre archivos de spec, la vista de progreso de tareas, y la interacción con el design document son inherentemente visuales. La CLI está optimizada para workflows más directos: chat, automatización, MCP, y headless.

¿Qué usar entonces?

Necesidad	Herramienta
Desarrollar features completas con specs	Kiro IDE
Chat agéntico desde la terminal	Kiro CLI
Automatización y CI/CD	Kiro CLI (headless)
Conectar herramientas externas vía MCP	Kiro CLI o Kiro IDE
Steering files para contexto de equipo	Kiro CLI o Kiro IDE
Troubleshooting rápido de AWS	Kiro CLI

Mi recomendación: usa ambos. Kiro IDE para desarrollo de features con specs, y Kiro CLI para todo lo que haces en la terminal — troubleshooting, automatización, operaciones, y CI/CD. Los steering files funcionan en ambos, así que tu contexto de proyecto se comparte.

MCP: lo que hace a Kiro CLI realmente poderoso

El Model Context Protocol es un estándar abierto que permite a agentes de IA conectarse de forma segura con herramientas externas, fuentes de datos y servicios. En la práctica esto significa que puedes extender las capacidades de Kiro CLI conectándolo a servidores MCP especializados.

Algunos ejemplos reales que ya existen:

Servidor MCP	Qué hace
CloudWatch MCP	Consulta métricas, logs y alarmas con lenguaje natural
Amazon MSK MCP	Administra clusters de Kafka — topics, configuraciones, health
AWS Diagram MCP	Genera diagramas de arquitectura AWS desde prompts
OpenSearch MCP	Busca índices, inspecciona estado del cluster, diagnósticos
Okta MCP	Gestión de identidades — usuarios, grupos, permisos
AWS Documentation MCP	Busca y lee documentación de AWS en contexto

La configuración de un servidor MCP se hace en ~/.kiro/settings/mcp.json. Una vez configurado, Kiro CLI tiene acceso a las herramientas del servidor y las usa automáticamente cuando son relevantes para tu pregunta.

Lo que hace esto particularmente poderoso es que puedes combinar múltiples servidores MCP. Imagina preguntarle a Kiro CLI: "¿por qué mi aplicación está lenta?" y que automáticamente consulte CloudWatch para métricas, OpenSearch para logs, y te dé un diagnóstico completo — todo desde tu terminal.

Los métodos de login — esto es clave

Acá es donde la cosa se pone interesante de verdad, porque Kiro CLI hereda el modelo de autenticación de Amazon Q Developer pero con matices que importan. Hay cuatro formas de conectarte, y cada una te da acceso a cosas diferentes:

1. Builder ID (Free) — para empezar rápido

Es la forma más simple. Te creas un AWS Builder ID gratis (con tu email, Google, Apple, GitHub o Amazon) y listo, ya puedes usar Kiro CLI y el IDE.

La limitación: tienes límites mensuales de uso y solo funciona en el IDE y la CLI. No tienes acceso a la consola de AWS ni a features avanzados. Pero para proyectos personales y exploración, es más que suficiente.

2. Builder ID + Pro — más límites, tu propia cuenta AWS

Acá es donde empieza a ponerse bueno. Puedes hacer upgrade de tu Builder ID al tier Pro conectándolo a tu propia cuenta de AWS. Esto te da límites de uso mucho más altos.

# Inicia chat con Kiro CLI
kiro-cli chat

# Dentro del chat, escribe:
/subscribe
# Esto abre la consola de AWS para confirmar la suscripción Pro

El punto clave es que con Builder ID Pro tienes límites más altos, pero no todas las features Pro. Algunas features avanzadas solo están disponibles vía IAM Identity Center.

3. IAM Identity Center (Free) — para equipos y organizaciones

Si tu empresa ya usa IAM Identity Center (antes AWS SSO), puedes autenticarte con tu identidad corporativa. Esto te da acceso a la consola de AWS, apps y websites de AWS — algo que Builder ID no puede hacer.

Ideal si tu admin ya configuró el Identity Center en la organización.

4. IAM Identity Center + Pro — el combo completo 🔥

Y acá está el gold standard. Tu admin te suscribe a Amazon Q Developer Pro vía IAM Identity Center, y tienes acceso a todo: CLI, IDE, consola, apps de AWS, features avanzados, límites altos, y control empresarial.

Lo que hace esto particularmente poderoso es que tu empresa tiene control total: puede suscribir usuarios en bulk, trackear uso, cancelar suscripciones, y tú como developer tienes la experiencia completa en todos los canales.

La guía rápida de acceso

🆓 Builder ID — Free

✅ CLI · ✅ IDE · ❌ Consola AWS · ❌ Features Pro
→ Para empezar rápido con proyectos personales

💰 Builder ID — Pro

✅ CLI · ✅ IDE · ❌ Consola AWS · ⚠️ Features Pro parciales
→ Límites más altos, pero no el suite completo

🏢 IAM Identity Center — Free

❌ CLI · ❌ IDE · ✅ Consola AWS · ❌ Features Pro
→ Solo consola, ideal si tu admin aún no activó Pro

🔥 IAM Identity Center — Pro

✅ CLI · ✅ IDE · ✅ Consola AWS · ✅ Features Pro completos
→ La experiencia completa — el gold standard

Mi recomendación sincera: si eres developer individual, empieza con Builder ID Free y cuando sientas los límites, haz upgrade a Pro con tu cuenta AWS. Si estás en una empresa, pídele a tu admin que configure IAM Identity Center con Pro — es la experiencia más completa y además puedes usarlo desde Kiro IDE con toda la potencia.

Headless Mode: Kiro CLI en CI/CD

Una capacidad nueva que no existía en Q Developer CLI es el modo headless — puedes ejecutar prompts de forma no interactiva usando API keys. Esto abre la puerta a integrar Kiro CLI en pipelines de CI/CD.

En la práctica esto significa que puedes automatizar tareas como:

Revisión de código automatizada en PRs
Generación de documentación en cada merge
Análisis de seguridad como paso del pipeline
Generación de tests para código nuevo

¿Debería migrar ya?

Sí, y sin dudarlo. Amazon Q Developer CLI ya no recibe actualizaciones activas. Todo el desarrollo está en Kiro CLI.

Pero más allá de la migración técnica, lo que me parece más valioso es el cambio de mentalidad que propone Kiro: dejar de usar la IA como un buscador glorificado y empezar a usarla como un agente que entiende tu contexto, se conecta a tus herramientas, y trabaja contigo en workflows reales.

La migración en sí es simple:

# 1. Instalar Kiro CLI
curl -fsSL https://cli.kiro.dev/install | bash

# 2. Donde antes usabas 'q chat', ahora usas:
kiro-cli chat

# 3. Donde usabas 'q translate', ahora:
kiro-cli translate "tu instrucción aquí"

El takeaway principal

El cambio de Amazon Q Developer CLI a Kiro CLI no es cosmético. Es una señal clara de hacia dónde va AWS con sus herramientas de desarrollo: agentes con contexto persistente, conectados a tus herramientas vía MCP, que entienden las convenciones de tu equipo vía steering files, y que pueden actuar — no solo responder.

Las capacidades clave que ganamos:

kiro-cli chat — Chat agéntico con capacidad de leer/escribir archivos y ejecutar comandos
kiro-cli translate — Traduce lenguaje natural a bash
kiro-cli inline — Sugerencias inteligentes mientras escribes
MCP Integration — Conecta herramientas externas (CloudWatch, MSK, OpenSearch, etc.)
Agent Steering — Dale contexto persistente a Kiro con las prácticas y convenciones de tu equipo
Custom Agents — Crea agentes especializados para tus workflows
Smart Hooks — Automatización pre/post comandos
Headless Mode — Integración con CI/CD vía API keys

Y un punto importante: Spec-Driven Development es exclusivo del Kiro IDE. Si quieres el workflow completo de specs (requirements → design → tasks → implementación), necesitas el IDE. La CLI es para chat, automatización, MCP, y operaciones desde la terminal. Ambos comparten steering files, así que tu contexto de proyecto funciona en los dos.

Mi recomendación: instala Kiro CLI, crea un steering file para tu proyecto más activo, conecta un servidor MCP relevante para tu stack, y experimenta. La curva de aprendizaje es corta y el salto de productividad es real.

Yo soy Carlos Cortez y esto es Breaking the Cloud — nos vemos pronto.

Sígueme en:

🔗 LinkedIn
🐦 X / Twitter
💻 GitHub
📝 Dev.to
🦸 AWS Community
✍️ Medium

How I Monitor AI Agents: CloudWatch for Infra, Arize Phoenix for Traces and OpenTelemetry, LLM-as-Judge for Quality

Carlos Cortez 🇵🇪 [AWS Hero] — Thu, 14 May 2026 00:56:26 +0000

How I Monitor My AI Agents: CloudWatch for Infra, Arize Phoenix for Traces, LLM-as-Judge for Quality

AI agents are not regular software. They reason, they call tools, they make decisions — and they can fail in ways that a simple health check will never catch. The response was technically successful, but was it actually helpful? The agent called the right tool, but did it interpret the result correctly? Traditional monitoring doesn't answer these questions.

That's why I built a three-layer observability stack for my AI agents, and today I'm walking you through exactly how it works.

📓 Full working notebook: All the code in this post is validated and executable in the companion Jupyter notebook — including setup, tracing, evals, and cleanup. here as well: https://github.com/breakingthecloud/observability-ai-agents-phoenix-otel-strands

The Problem with Monitoring AI Agents

Here's the thing: when your agent answers "I don't have weather data for Paris" — is that a failure? Technically no, the agent ran fine. But from a user perspective, it's a miss. Traditional monitoring would show 200 OK, low latency, zero errors. Everything looks green. But the user didn't get what they needed.

You need three layers of observability to actually understand what's happening:

User Query → Strands Agent → Tool Calls → Bedrock (Claude)
     ↓              ↓              ↓
  Phoenix      CloudWatch      Phoenix Evals
 (AI traces)  (infra metrics)  (quality scores)

Layer	Tool	What it answers
AI Traces	Arize Phoenix	What did the agent think? Which tools did it call? What was the full LLM input/output?
Infrastructure	Amazon CloudWatch	Is the system healthy? How fast? How much is it costing me?
Quality Evals	Phoenix + LLM-as-Judge	Was the response actually good? Helpful? Accurate?

The Stack

Strands Agents SDK — AWS's open-source framework for building agents
Amazon Bedrock — Claude Sonnet 4.6 as the foundation model
Arize Phoenix — Open-source AI observability, runs locally, zero accounts needed
Amazon CloudWatch — Metrics, alarms, dashboards
OpenTelemetry — The glue that connects everything

What's interesting is that Phoenix runs entirely on your machine — localhost:6006. No cloud accounts, no API keys for the observability layer. You get a full tracing UI for free.

Layer 1: AI Traces with Arize Phoenix

The first thing you need is visibility into what your agent is actually doing. Not just "it responded in 2 seconds" but the full reasoning chain: what the LLM received, what it decided, which tools it called, and what it returned.

Setting Up the Tracing Pipeline

Three steps: launch Phoenix, configure OpenTelemetry, instrument Bedrock.

import phoenix as px
from opentelemetry import trace as trace_api
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.bedrock import BedrockInstrumentor

# 1. Launch Phoenix locally
session = px.launch_app()  # UI at http://localhost:6006

# 2. Configure OTel to send traces to Phoenix
tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(
    SimpleSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:6006/v1/traces"))
)
trace_api.set_tracer_provider(tracer_provider)

# 3. Auto-instrument all Bedrock API calls
BedrockInstrumentor().instrument(tracer_provider=tracer_provider)

That's it. Every Bedrock call your agent makes is now traced automatically. No decorators on your business logic, no manual span creation. OpenInference handles it.

Building the Agent

The agent itself is straightforward with Strands:

import boto3
from strands import Agent
from strands.models.bedrock import BedrockModel

session = boto3.Session(profile_name="cc", region_name="us-east-1")

def get_weather(city: str) -> str:
    """Get current weather for a city."""
    weather_data = {
        "Lima": "☀️ 22°C, clear skies",
        "New York": "🌧️ 15°C, rainy",
        "Tokyo": "⛅ 18°C, partly cloudy",
    }
    return weather_data.get(city, f"Weather data not available for {city}")

model = BedrockModel(model_id="us.anthropic.claude-sonnet-4-6", boto_session=session)

agent = Agent(
    model=model,
    tools=[get_weather],
    system_prompt="You are a helpful weather assistant. Use the get_weather tool."
)

Now when you run agent("What's the weather in Lima and Tokyo?"), Phoenix captures the entire trace tree: the agent span, the LLM calls, the tool invocations, the final response. All visible in the UI at localhost:6006.

Exploring Traces Programmatically

You don't have to use the UI. Phoenix exposes everything as DataFrames:

from phoenix.client import Client

traces_df = Client().spans.get_spans_dataframe()
traces_df["latency_ms"] = (traces_df["end_time"] - traces_df["start_time"]).dt.total_seconds() * 1000

print(f"Total spans captured: {len(traces_df)}")
traces_df[["name", "span_kind", "latency_ms", "status_code"]].head(10)

This gives you every span — agent, LLM, tool — with timing, status, and the full input/output attributes. Perfect for building custom analytics or feeding into your own dashboards.

Layer 2: Infrastructure Monitoring with CloudWatch

Phoenix tells you what the agent is thinking. CloudWatch tells you if the system is healthy. Different questions, both critical.

The AgentMonitor Class

I built a simple wrapper that publishes four metrics per agent invocation:

cloudwatch = session.client("cloudwatch", region_name="us-east-1")

class AgentMonitor:
    def __init__(self, namespace="AI/Agents"):
        self.namespace = namespace
        self.cw = cloudwatch

    def track(self, agent_name: str, latency_ms: float, tokens: int,
              success: bool, tool_calls: int = 0):
        metrics = [
            {"MetricName": "Latency", "Value": latency_ms, "Unit": "Milliseconds"},
            {"MetricName": "TokensUsed", "Value": tokens, "Unit": "Count"},
            {"MetricName": "Success", "Value": 1 if success else 0, "Unit": "Count"},
            {"MetricName": "ToolCalls", "Value": tool_calls, "Unit": "Count"},
        ]
        dims = [{"Name": "AgentName", "Value": agent_name}]
        for m in metrics:
            m["Dimensions"] = dims
        self.cw.put_metric_data(Namespace=self.namespace, MetricData=metrics)

Usage is clean — wrap your agent call:

monitor = AgentMonitor()
start = time.time()
try:
    result = agent("What's the weather in Lima?")
    latency = (time.time() - start) * 1000
    monitor.track("weather-agent", latency, tokens=150, success=True, tool_calls=1)
except Exception as e:
    latency = (time.time() - start) * 1000
    monitor.track("weather-agent", latency, tokens=0, success=False, tool_calls=0)

Smart Alarms

Two alarms that catch the most common issues:

# Alert when response time is consistently high
cloudwatch.put_metric_alarm(
    AlarmName="Agent-High-Latency",
    MetricName="Latency", Namespace="AI/Agents",
    Statistic="Average", Period=300, EvaluationPeriods=3,
    Threshold=10000.0,  # 10 seconds
    ComparisonOperator="GreaterThanThreshold",
    Dimensions=[{"Name": "AgentName", "Value": "weather-agent"}],
)

# Alert when error rate exceeds 5%
cloudwatch.put_metric_alarm(
    AlarmName="Agent-High-Error-Rate",
    MetricName="Success", Namespace="AI/Agents",
    Statistic="Average", Period=300, EvaluationPeriods=2,
    Threshold=0.95,
    ComparisonOperator="LessThanThreshold",
    Dimensions=[{"Name": "AgentName", "Value": "weather-agent"}],
)

The concept is straightforward: latency catches performance degradation, error rate catches reliability issues. These two alarms alone will catch 80% of production problems.

Layer 3: LLM-as-Judge Evals

This is the layer most people skip — and it's the most important one. Your agent can be fast, reliable, and still give terrible answers. You need automated quality evaluation.

The idea: use another LLM to judge the quality of your agent's responses. It's not perfect, but it's infinitely better than no evaluation at all.

Setting Up the Evaluator

Phoenix evals v3 uses a provider-based LLM wrapper. For Bedrock, it goes through litellm:

from phoenix.evals import LLM, create_evaluator, evaluate_dataframe
from phoenix.client import Client
import pandas as pd

# Get LLM spans from Phoenix
spans_df = Client().spans.get_spans_dataframe()
llm_spans = spans_df[spans_df["span_kind"] == "LLM"].copy()

# Build eval dataframe
eval_data = pd.DataFrame({
    "input": llm_spans["attributes.input.value"].fillna("").values,
    "output": llm_spans["attributes.output.value"].fillna("").values,
})
eval_data = eval_data[eval_data["output"].str.len() > 0].reset_index(drop=True)

# Create the judge
eval_model = LLM(provider="bedrock", model="us.anthropic.claude-sonnet-4-6")

@create_evaluator(name="helpfulness", source="llm")
def helpfulness(input: str, output: str) -> float:
    """Rate how helpful the agent response is on a scale of 0 to 1."""
    prompt = (
        f"Rate the helpfulness of this AI response on a scale of 0.0 to 1.0.\n"
        f"User asked: {input}\n"
        f"AI responded: {output}\n"
        f"Return ONLY a number between 0.0 and 1.0."
    )
    result = eval_model.generate_text(prompt=prompt)
    try:
        return float(result.strip())
    except ValueError:
        return 0.5

# Run evaluation
results = evaluate_dataframe(dataframe=eval_data, evaluators=[helpfulness])

The cool part here is the @create_evaluator decorator — it turns a simple function into a full evaluator that Phoenix understands. You can create as many as you need: helpfulness, accuracy, safety, tone, whatever matters for your use case.

Pushing Scores Back to Phoenix

The evaluation results are useful in a DataFrame, but they're even more useful when attached to the actual traces in Phoenix:

import json as _json

score_col = [c for c in results.columns if "_score" in c][0]
scores = results[score_col].apply(
    lambda x: _json.loads(x).get("value", 0) if isinstance(x, str) else 0
)

annotations = pd.DataFrame({
    "span_id": llm_spans["context.span_id"].values[:len(scores)],
    "score": scores.values,
    "label": scores.apply(lambda s: "good" if s >= 0.7 else "needs_review").values,
    "explanation": [f"Helpfulness score: {s:.2f}" for s in scores.values],
})

Client().spans.log_span_annotations_dataframe(
    dataframe=annotations,
    annotation_name="helpfulness",
    annotator_kind="LLM",
)

Now when you open Phoenix UI and click on any LLM span, you see the helpfulness score right there in the Annotations tab. Traces + quality scores in one place.

What This Costs

One question I always get: what does this cost to run?

Component	Where	Cost
Phoenix	Your machine (localhost)	$0
Bedrock (agent calls)	AWS, pay-per-request	~$0.003 per query
Bedrock (eval judge)	AWS, pay-per-request	~$0.003 per eval
CloudWatch alarms	AWS	~$0.20/month
CloudWatch custom metrics	AWS	~$0.30/month

For development and testing, you're looking at less than $1/month for the AWS side. Phoenix is completely free and local.

The Main Takeaway

Observability for AI agents requires thinking in three layers:

Traces (Phoenix) — What is the agent doing? What's the full reasoning chain?
Infra metrics (CloudWatch) — Is the system healthy? Fast? Within budget?
Quality evals (LLM-as-Judge) — Are the responses actually good?

Most teams only do layer 2. Some add layer 1. Almost nobody does layer 3 — and that's where the real insights are. A fast, reliable agent that gives bad answers is worse than a slow one that gives good answers, because you won't even know there's a problem.

My advice: start with Phoenix traces (it's free and local), add CloudWatch for the basics (latency, errors, tokens), and then build at least one LLM-as-Judge evaluator for whatever quality dimension matters most to your users. You can set this up in an afternoon and it will save you weeks of debugging blind.

Connect with me:

LinkedIn - Let's discuss AI observability and agent architectures
X/Twitter - Follow for AWS, GenAI, and agentic AI updates
GitHub - Check out the full notebook and more
Dev.to - More technical deep-dives
AWS Community - Join the conversation

I'm Carlos Cortez, this is Breaking the Cloud, and today we made our agents observable. See you in the next one!