DEV Community: Carlos Cortez 🇵🇪 [AWS Hero]

I Built a Minecraft Mod Where Every Sword is an AWS Service — Here's How We Coded It with AI

Carlos Cortez 🇵🇪 [AWS Hero] — Tue, 05 May 2026 01:43:58 +0000

I Built a Minecraft Mod Where Every Sword is an AWS Service — Here's How We Coded It with AI

What happens when a cloud engineer picks up Minecraft modding for the first time? You get swords that invoke Lambda functions, store items in S3 buckets, and auto-scale damage like EC2 instances. Today we're going deep into how I built AWS Swords — a Fabric mod for Minecraft 1.21.1 where every weapon is inspired by a real AWS service, and every ability maps to actual cloud computing concepts.

The concept is straightforward: take the services we use every day in the cloud and turn them into Minecraft weapons with abilities that actually make sense. Lambda is serverless and ephemeral? The sword summons temporary allies that disappear after a few seconds. S3 stores objects? The sword absorbs items from the ground and retrieves them later. EC2 scales? The sword stacks damage the more you hit.

And the best part — I coded this entire mod alongside an AI agent. Every class, every texture animation, every ability was a conversation. Let me show you how.

📓 Full source code: The complete mod is available on CurseForge — clone it, build it, swing some cloud-powered swords.

Why AWS Swords?

I've been a modded Minecraft fan for years. Tech mods and RPG packs are my thing — Applied Energistics 2, Tech Reborn, Mekanism, Create, you name it. There's something deeply satisfying about building complex systems inside a game, automating everything, and watching it all come together. If you've ever spent an entire weekend wiring up an ME system or designing a Mekanism ore processing chain, you know exactly what I'm talking about.

So when I started thinking about a side project that combined my two worlds — cloud computing and gaming — the idea hit me: what if there was a tech mod, but for cloud computing? Not pipes and machines, but AWS services as weapons with abilities that mirror what the real services do. And I wanted to go all the way with it — build it for Minecraft 1.21.1, publish it officially on CurseForge, and include it in my current modded Minecraft series.

That's how AWS Swords was born. And what's interesting is how naturally AWS service behaviors map to game mechanics:

AWS Lambda → Ephemeral compute → Temporary allies that spawn and despawn
Amazon S3 → Object storage → A sword that absorbs and retrieves items
Amazon EC2 → Auto Scaling + Spot Instances → Stacking damage + random crits

In practice, this means that every sword teaches you something about the service it represents, just by playing with it.

The Architecture

Here's the mod structure — and yes, it looks a lot like a microservices architecture:

aws-swords-mod/
├── src/main/java/com/awsswords/
│   ├── AwsSwordsMod.java              # Entrypoint (the main() of mods)
│   ├── client/
│   │   └── AwsSwordsClient.java       # Client-side particles
│   ├── entity/
│   │   └── LambdaMinionManager.java   # Lifecycle manager for Lambda invocations
│   └── item/
│       ├── ModItems.java              # Item registry (like a service catalog)
│       ├── ability/
│       │   └── SwordAbility.java      # Interface — the contract every ability follows
│       └── sword/
│           ├── BaseSword.java         # Abstract base — cooldowns, tooltips, particles
│           ├── LambdaSword.java       # ⚡ Invoke: 1-3 temporary allies
│           ├── GreatswordOfLambda.java # ⚔ Mass Invoke: 3-5 holy allies
│           ├── S3Sword.java           # ☁ PutObject / GetObject item storage
│           └── EC2Sword.java          # ⚡ Auto Scaling + Spot Instance crits
├── src/main/resources/
│   ├── fabric.mod.json
│   └── assets/awsswords/
│       ├── lang/                      # en_us.json + es_es.json
│       ├── models/item/               # JSON models per sword
│       └── textures/item/             # Animated PNGs + .mcmeta
└── build.gradle

The design follows a pattern any cloud architect would recognize: an interface defines the contract (SwordAbility), an abstract base class handles shared behavior (BaseSword), and each implementation is independent (one class per sword). Loose coupling, high cohesion. Just like good microservices.

The Tech Stack

Component	Version
Minecraft	1.21.1
Fabric Loader	0.19.2
Fabric API	0.109.0+1.21.1
Fabric Loom	1.9
Java	21
Gradle	8.12

Step 1: The Foundation — SwordAbility Interface

Every sword ability in the mod implements this interface. It's the contract — if you want to be an AWS sword, you need to define what happens on use, how long the cooldown is, and what the tooltip says.

public interface SwordAbility {
    void onUse(PlayerEntity player, World world, ItemStack stack);
    default void onAttack(PlayerEntity player, Entity target, ItemStack stack) {}
    int getCooldownTicks();
    Text getDescription();
}

What caught my attention is how clean this keeps things. Adding a new sword means implementing this interface and wiring it up. No giant switch statements, no spaghetti. The onAttack default method is there for swords like EC2 that need hit-based mechanics instead of right-click abilities.

Step 2: BaseSword — The Shared Infrastructure

The BaseSword abstract class handles everything that's common across swords: Netherite-tier stats, cooldown management, tooltip rendering, and the hook for ambient particles.

public abstract class BaseSword extends SwordItem {
    private final SwordAbility ability;
    private final String lore;

    protected BaseSword(SwordAbility ability, String lore, Item.Settings settings) {
        super(ToolMaterials.NETHERITE, settings.attributeModifiers(
                SwordItem.createAttributeModifiers(ToolMaterials.NETHERITE, 3, -2.4f)));
        this.ability = ability;
        this.lore = lore;
    }

    @Override
    public TypedActionResult<ItemStack> use(World world, PlayerEntity player, Hand hand) {
        ItemStack stack = player.getStackInHand(hand);
        if (!world.isClient && !player.getItemCooldownManager().isCoolingDown(this)) {
            ability.onUse(player, world, stack);
            player.getItemCooldownManager().set(this, ability.getCooldownTicks());
            return TypedActionResult.success(stack);
        }
        return TypedActionResult.pass(stack);
    }

    @Override
    public void appendTooltip(ItemStack stack, TooltipContext context, List<Text> tooltip, TooltipType type) {
        tooltip.add(ability.getDescription().copy().formatted(Formatting.GOLD));
        tooltip.add(Text.literal("Cooldown: " + ability.getCooldownTicks() / 20 + "s").formatted(Formatting.GRAY));
        tooltip.add(Text.literal(lore).formatted(Formatting.DARK_PURPLE, Formatting.ITALIC));
    }
}

Think of BaseSword as a base AMI — it has everything pre-configured, and each sword just layers its unique behavior on top. The use() method delegates to the ability, handles cooldowns, and makes sure we only run server-side logic (no duplicate execution on the client).

Step 3: Sword of Lambda — Serverless Allies

This is where it gets fun. AWS Lambda runs code on demand, scales automatically, and the compute is ephemeral — it disappears when the function finishes. So the Sword of Lambda invokes 1-3 baby zombie allies that fight for you and despawn after 5 seconds.

public class LambdaSword extends BaseSword {
    public LambdaSword(Item.Settings settings) {
        super(new InvokeAbility(), "Runs on demand. No servers required.", settings);
    }

    @Override
    protected ParticleEffect getAmbientParticle() {
        return ParticleTypes.ENCHANTED_HIT;
    }

    @Override
    public boolean hasGlint(ItemStack stack) { return true; }

    private static class InvokeAbility implements SwordAbility {
        @Override
        public void onUse(PlayerEntity player, World world, ItemStack stack) {
            if (!(world instanceof ServerWorld server)) return;

            int count = 1 + world.random.nextInt(3);
            world.playSound(null, player.getBlockPos(), SoundEvents.ENTITY_EVOKER_CAST_SPELL,
                    SoundCategory.PLAYERS, 1f, 1.5f);

            for (int i = 0; i < count; i++) {
                ZombieEntity minion = new ZombieEntity(EntityType.ZOMBIE, world);
                double angle = (2 * Math.PI * i) / count;
                double x = player.getX() + Math.cos(angle) * 2;
                double z = player.getZ() + Math.sin(angle) * 2;
                minion.refreshPositionAndAngles(x, player.getY(), z,
                        (float)(angle * 180 / Math.PI), 0);
                minion.setBaby(true);
                minion.setCustomName(Text.literal("λ Invocation").formatted(Formatting.GOLD));
                minion.setCustomNameVisible(true);
                server.spawnEntity(minion);
                server.spawnParticles(ParticleTypes.FLAME,
                        x, player.getY() + 0.5, z, 10, 0.3, 0.5, 0.3, 0.02);
                LambdaMinionManager.track(minion, 100); // 5 seconds
            }

            player.sendMessage(Text.literal(
                    "§6⚡ Lambda Invoke: " + count + " function" +
                    (count > 1 ? "s" : "") + " deployed!"), true);
        }

        @Override
        public int getCooldownTicks() { return 300; } // 15 seconds

        @Override
        public Text getDescription() {
            return Text.literal("⚡ Invoke — Summon 1-3 temporary allies");
        }
    }
}

A few things to note:

The minions spawn in a circle around the player using trigonometry — just like Lambda functions spinning up across availability zones
Each minion gets a custom name tag: λ Invocation in gold text
The LambdaMinionManager.track(minion, 100) call registers the entity for automatic cleanup after 100 ticks (5 seconds)
The evoker cast spell sound gives it that magical "deploying to the cloud" feel

The LambdaMinionManager — Lifecycle Management

This is the garbage collector for our serverless functions. It uses Fabric's ServerTickEvents to count down each minion's remaining ticks and despawn them with a puff of smoke:

public class LambdaMinionManager {
    private static final List<TrackedMinion> minions = new ArrayList<>();

    public static void register() {
        ServerTickEvents.END_SERVER_TICK.register(server -> {
            Iterator<TrackedMinion> it = minions.iterator();
            while (it.hasNext()) {
                TrackedMinion tracked = it.next();
                tracked.ticksLeft--;
                if (tracked.ticksLeft <= 0 || !tracked.entity.isAlive()) {
                    if (tracked.entity.isAlive()) {
                        if (tracked.entity.getWorld() instanceof ServerWorld sw) {
                            sw.spawnParticles(ParticleTypes.SMOKE,
                                    tracked.entity.getX(), tracked.entity.getY() + 0.5,
                                    tracked.entity.getZ(), 15, 0.3, 0.5, 0.3, 0.05);
                        }
                        tracked.entity.discard();
                    }
                    it.remove();
                }
            }
        });
    }

    public static void track(ZombieEntity entity, int ticks) {
        minions.add(new TrackedMinion(entity, ticks));
    }
}

Think of it as CloudWatch monitoring your Lambda executions — when the timeout hits, the function gets terminated. Clean, predictable lifecycle management.

Step 4: Greatsword of Lambda — Mass Invoke

Same concept, bigger scale. The Greatsword is the two-handed variant — slower attack speed (0.8 vs 1.6), higher damage (+15 vs +11), and it summons 3-5 allies that last 8 seconds instead of 5. It's like going from a single Lambda invocation to a fan-out pattern.

// attackDamage +7 (vs +3 normal), attackSpeed -3.2 (very slow, greatsword feel)
super(ToolMaterials.NETHERITE, settings.attributeModifiers(
        SwordItem.createAttributeModifiers(ToolMaterials.NETHERITE, 7, -3.2f)));

The minions get purple soul fire particles instead of regular flames, and their name tags read λ Holy Invocation — because when you're invoking 5 functions at once, it's practically a religious experience.

Step 5: Sword of S3 — Object Storage in Your Inventory

This one is my favorite. Amazon S3 stores and retrieves objects. So the Sword of S3 does exactly that:

Right-click → PutObject: absorbs the nearest item from the ground (4-block radius) into a virtual bucket
Shift + Right-click → GetObject: retrieves the last stored item back to your inventory
Max capacity: 5 objects (it's a bucket, not a data lake)
Persistence: Items are stored in the sword's NBT data — they survive death

private TypedActionResult<ItemStack> putObject(ServerWorld world, PlayerEntity player, ItemStack sword) {
    NbtList bucket = getBucket(sword);
    if (bucket.size() >= MAX_OBJECTS) {
        player.sendMessage(Text.literal("§c✗ Bucket full! (5/5 objects)"), true);
        return TypedActionResult.fail(sword);
    }

    List<ItemEntity> items = world.getEntitiesByClass(ItemEntity.class,
            new Box(player.getBlockPos()).expand(4), e -> !e.isRemoved());

    if (items.isEmpty()) {
        player.sendMessage(Text.literal("§7No items nearby to upload"), true);
        return TypedActionResult.fail(sword);
    }

    // Find nearest item
    ItemEntity nearest = items.get(0);
    double minDist = player.squaredDistanceTo(nearest);
    for (ItemEntity ie : items) {
        double d = player.squaredDistanceTo(ie);
        if (d < minDist) { minDist = d; nearest = ie; }
    }

    // Store item as NBT
    ItemStack picked = nearest.getStack().copy();
    NbtCompound itemNbt = new NbtCompound();
    itemNbt.put("item", picked.encodeAllowEmpty(world.getRegistryManager()));
    bucket.add(itemNbt);
    setBucket(sword, bucket);

    nearest.discard();
    world.playSound(null, player.getBlockPos(), SoundEvents.ENTITY_ENDERMAN_TELEPORT,
            SoundCategory.PLAYERS, 0.7f, 1.5f);
    world.spawnParticles(ParticleTypes.PORTAL,
            nearest.getX(), nearest.getY() + 0.5, nearest.getZ(), 20, 0.3, 0.3, 0.3, 0.5);

    player.sendMessage(Text.literal("§b☁ PutObject: " + picked.getName().getString() +
            " uploaded (" + bucket.size() + "/" + MAX_OBJECTS + ")"), true);
    return TypedActionResult.success(sword);
}

The cool part here is the NBT storage pattern. In Minecraft 1.21.1, items use the new DataComponentTypes.CUSTOM_DATA system instead of the old direct NBT access. The sword stores a NbtList called s3bucket inside its custom data component — each entry is a serialized ItemStack. It's literally key-value storage, just like S3.

The visual feedback sells it: Portal particles on upload (your item is going to the cloud), Reverse Portal particles on download (it's coming back), and the Enderman teleport sound for both operations. Because if S3 had a sound, it would be a teleport.

The lore text? "11 nines of durability." — because S3 offers 99.999999999% durability, and that joke was too good to pass up.

Step 6: Sword of EC2 — Auto Scaling Damage

EC2 is all about compute that scales. The Sword of EC2 has two mechanics:

Auto Scaling (Active): Consecutive hits within 4 seconds stack damage — +10% per hit, up to +50% at 5 stacks. Stop hitting for 4 seconds and it resets. Just like an Auto Scaling group ramping up instances under load.

Spot Instance (Passive): 20% chance on every hit to land a critical strike with electric spark particles and a lightning sound. Because Spot Instances give you extra compute at a discount, but they can be interrupted at any time — you never know when you'll get one.

@Override
public boolean postHit(ItemStack stack, LivingEntity target, LivingEntity attacker) {
    if (!(attacker instanceof PlayerEntity player) || attacker.getWorld().isClient) {
        return super.postHit(stack, target, attacker);
    }

    ServerWorld world = (ServerWorld) attacker.getWorld();
    long now = world.getTime();
    UUID id = player.getUuid();

    // Auto Scaling: increment stacks
    StackData data = playerStacks.computeIfAbsent(id, k -> new StackData());
    if (now - data.lastHitTick > STACK_TIMEOUT_TICKS) {
        data.stacks = 0; // reset — scale-in after idle
    }
    data.lastHitTick = now;
    if (data.stacks < MAX_STACKS) data.stacks++;

    float bonus = data.stacks * 0.10f;
    float bonusDamage = target.getMaxHealth() > 0 ? stack.getDamage() * bonus : 0;
    if (bonusDamage > 0) {
        target.damage(world.getDamageSources().playerAttack(player), bonusDamage);
    }

    // Spot Instance: 20% crit chance
    boolean spotCrit = world.random.nextFloat() < 0.20f;
    if (spotCrit) {
        target.damage(world.getDamageSources().playerAttack(player), 4.0f);
        world.spawnParticles(ParticleTypes.ELECTRIC_SPARK,
                target.getX(), target.getY() + 1, target.getZ(), 20, 0.5, 0.8, 0.5, 0.1);
        world.playSound(null, target.getBlockPos(), SoundEvents.ENTITY_LIGHTNING_BOLT_IMPACT,
                SoundCategory.PLAYERS, 0.4f, 1.8f);
    }

    // More particles at higher stacks — visual scaling
    world.spawnParticles(ParticleTypes.SOUL_FIRE_FLAME,
            target.getX(), target.getY() + 1, target.getZ(),
            2 + data.stacks * 3, 0.3, 0.5, 0.3, 0.02);

    int pct = (int)(bonus * 100);
    String msg = "§b⚡ Auto Scaling: " + data.stacks + "/" + MAX_STACKS + " (+" + pct + "%)";
    if (spotCrit) msg += " §e⚡ SPOT CRIT!";
    player.sendMessage(Text.literal(msg), true);

    return super.postHit(stack, target, attacker);
}

Notice how the particle count scales with the stack count (2 + data.stacks * 3). At max stacks you get 17 soul fire particles per hit — the visual feedback tells you exactly how scaled up you are. The EC2 sword uses postHit instead of onUse because its mechanics are attack-based, not right-click-based. Different interaction model, same interface contract.

Step 7: Animated Textures and Visual Polish

Every sword has an 8-frame animated texture with interpolation. In Minecraft, you achieve this with a sprite sheet PNG (16x128 for 8 frames of 16x16) and a .mcmeta file:

{
  "animation": {
    "frametime": 3,
    "interpolate": true
  }
}

The frametime: 3 means each frame lasts 3 ticks (0.15 seconds), and interpolate: true smoothly blends between frames. Combined with hasGlint() → true on every sword, you get that enchanted shimmer on top of the animation.

The color palettes map to the services:

Lambda: Orange/purple with fire and gems
Greatsword of Lambda: Purple/pink holy warrior energy
S3: Green/teal with teal gems and green aura
EC2: Blue/orange katana-style with electric sparks

Step 8: Client-Side Particles

The AwsSwordsClient class adds ambient particles when you're holding any AWS sword. It runs client-side only (no server overhead) and spawns subtle enchanted hit particles every 6 ticks:

public class AwsSwordsClient implements ClientModInitializer {
    @Override
    public void onInitializeClient() {
        ClientTickEvents.END_CLIENT_TICK.register(client -> {
            ClientPlayerEntity player = client.player;
            if (player == null || !(player.getMainHandStack().getItem() instanceof BaseSword)) return;

            if (player.getWorld().getTime() % 6 == 0) {
                double x = player.getX() + (player.getRandom().nextDouble() - 0.5) * 0.5;
                double y = player.getY() + 0.8 + player.getRandom().nextDouble() * 0.5;
                double z = player.getZ() + (player.getRandom().nextDouble() - 0.5) * 0.5;
                player.getWorld().addParticle(ParticleTypes.ENCHANTED_HIT, x, y, z, 0, 0.02, 0);
            }
        });
    }
}

It's a small detail, but it makes the swords feel alive. You know you're holding something special.

Step 9: Registration and Localization

The ModItems class registers all swords with Fabric's registry and adds them to the Combat creative tab:

public class ModItems {
    public static final Item SWORD_OF_LAMBDA = register("sword_of_lambda",
            new LambdaSword(new Item.Settings()));
    public static final Item GREATSWORD_OF_LAMBDA = register("greatsword_of_lambda",
            new GreatswordOfLambda(new Item.Settings()));
    public static final Item SWORD_OF_S3 = register("sword_of_s3",
            new S3Sword(new Item.Settings()));
    public static final Item SWORD_OF_EC2 = register("sword_of_ec2",
            new EC2Sword(new Item.Settings()));

    private static Item register(String name, Item item) {
        return Registry.register(Registries.ITEM, Identifier.of(AwsSwordsMod.MOD_ID, name), item);
    }

    public static void register() {
        ItemGroupEvents.modifyEntriesEvent(ItemGroups.COMBAT).register(entries -> {
            entries.add(SWORD_OF_LAMBDA);
            entries.add(GREATSWORD_OF_LAMBDA);
            entries.add(SWORD_OF_S3);
            entries.add(SWORD_OF_EC2);
        });
    }
}

And because this is Breaking the Cloud, we ship with English and Spanish localization:

// en_us.json
{
  "item.awsswords.sword_of_lambda": "Sword of Lambda",
  "item.awsswords.greatsword_of_lambda": "Greatsword of Lambda",
  "item.awsswords.sword_of_s3": "Sword of S3",
  "item.awsswords.sword_of_ec2": "Sword of EC2"
}

// es_es.json
{
  "item.awsswords.sword_of_lambda": "Espada de Lambda",
  "item.awsswords.greatsword_of_lambda": "Gran Espada de Lambda",
  "item.awsswords.sword_of_s3": "Espada de S3",
  "item.awsswords.sword_of_ec2": "Espada de EC2"
}

Building and Running

Two scripts handle everything:

# Build the mod JAR
#!/bin/bash
export JAVA_HOME=/opt/homebrew/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home
export PATH="$JAVA_HOME/bin:$PATH"
cd "$(dirname "$0")/aws-swords-mod" && ./gradlew remapJar

# Run Minecraft with the mod loaded
#!/bin/bash
export JAVA_HOME=/opt/homebrew/opt/openjdk@21/libexec/openjdk.jdk/Contents/Home
export PATH="$JAVA_HOME/bin:$PATH"
cd "$(dirname "$0")/aws-swords-mod" && ./gradlew runClient

The remapJar task is Fabric-specific — it remaps your compiled classes from development mappings (Yarn) to production mappings (intermediary) so the mod works in a real Minecraft installation. The output lands in build/libs/aws-swords-0.1.0.jar.

Coding with AI — How We Actually Built This

Here's the part that makes this project different. I didn't build this mod alone — I built it in conversation with an AI agent. Every single file in this project was a collaboration.

The workflow looked like this:

I described the vision: "I want a Minecraft mod where swords have AWS service abilities"
We wrote a Statement of Work together: Service-to-ability mappings, damage values, cooldowns, visual effects
We coded iteratively: One sword at a time, testing in-game between each one
The AI handled the boilerplate: Gradle config, Fabric mod JSON, registry patterns, NBT serialization
I made the design decisions: Which services to include, how abilities should feel, what the lore text should say

The thing is, the AI was particularly good at the parts that would have taken me hours to figure out as a first-time Minecraft modder — things like the DataComponentTypes.CUSTOM_DATA pattern for NBT storage in 1.21.1, the ServerTickEvents hook for entity lifecycle management, and the TypedActionResult return types for item interactions.

But the creative decisions — Lambda should summon baby zombies because they're small and ephemeral, S3 should use portal particles because items are going to "the cloud", EC2's lore should be "Choose your instance type wisely" — those were all human.

My advice for anyone trying this: use AI for the mechanics, bring the creativity yourself.

Lessons Learned

Challenge	Solution
First time with Fabric modding	AI agent handled the boilerplate, I focused on game design
NBT storage in 1.21.1	New `DataComponentTypes.CUSTOM_DATA` system, not the old direct NBT
Entity lifecycle (Lambda minions)	Custom tick-based manager with `ServerTickEvents`
Client vs Server logic	`world.isClient` checks everywhere — particles client-side, logic server-side
Animated textures	8-frame sprite sheets + `.mcmeta` with `interpolate: true`
Greatsword balance	Higher damage (+7 vs +3) but much slower speed (-3.2 vs -2.4)
S3 item persistence	NBT serialization with `encodeAllowEmpty` + `ItemStack.fromNbt`
EC2 stack tracking	Per-player `HashMap<UUID, StackData>` with tick-based timeout

What's Next

The SoW has 3 more swords planned for the MVP:

Sword of CloudWatch — "Describe Alarms": reveals all mobs in 30 blocks, showing HP and active effects. Like a scanner. "Observable by default."
Sword of IAM — "Deny Policy": applies a debuff that blocks mob special abilities (no teleport for Endermen, no explosion for Creepers). "Explicit deny always wins."
Sword of DynamoDB — "Burst Capacity": first 5 hits in 3 seconds deal increasing damage, then you enter "throttle" mode with reduced damage. "Single-digit millisecond latency."

And Phase 2 includes SNS, SQS, Route 53, CloudFront, ECS, EKS, RDS, Kinesis, Step Functions, and Bedrock. Plus a rune and socket system. The roadmap is ambitious, but we're building it one sword at a time.

The Main Takeaway

You don't need to be a Minecraft modding expert to build a mod. You don't need to be a game designer to create fun mechanics. What you need is a clear vision, a good collaborator (AI or human), and the willingness to iterate.

The truth is, this project taught me more about Fabric's internals in a few hours than any tutorial could have. When you're building something you actually want to play, the learning happens naturally.

And if you're a cloud engineer who's ever wondered what your favorite AWS service would look like as a Minecraft weapon — now you know.

Connect with me:

LinkedIn - Let's discuss cloud architecture and Minecraft mods
X/Twitter - Follow for AWS, GenAI, and now gaming updates
GitHub - Check out the full mod source code
Dev.to - More technical deep-dives
AWS Community - Join the conversation

I'm Carlos Cortez, this is Breaking the Cloud, and today we broke it with diamond swords. See you in the next one!

How I Deployed CrewAI Multi-Agent Teams on Amazon Bedrock AgentCore Without Writing a Single Dockerfile

Carlos Cortez 🇵🇪 [AWS Hero] — Wed, 29 Apr 2026 00:49:46 +0000

How I Deployed CrewAI Multi-Agent Teams on Amazon Bedrock AgentCore Without Writing a Single Dockerfile

Building AI agents is exciting, but deploying them to production? That's where the real magic happens. Today we're diving deep into Amazon Bedrock AgentCore — AWS's serverless runtime for AI agents that eliminates all the infrastructure headaches.

the idea here is simple: you build your CrewAI multi-agent teams locally, test them thoroughly, and then deploy to AgentCore with the AgentCore CLI. No Dockerfiles, no scaling concerns, no operational overhead.

What makes this particularly interesting is how AgentCore bridges the gap between local development and production deployment. You write your agent code once, test it on localhost:8080, and then deploy it to a fully managed serverless environment available in 15 AWS regions.

📓 Full working notebook: All the code in this post is validated and executable in the companion Jupyter notebook — including deploy and cleanup.

Here as well in my github link below: https://github.com/breakingthecloud/crewai-agentcore-deployment/blob/main/bedrock-agentcore-crewai-validation.ipynb

Why AgentCore Changes the Game

Traditional AI agent deployment involves containers, load balancers, auto-scaling groups, and a whole infrastructure stack. AgentCore flips this model:

Serverless by design: No servers to manage, automatic scaling
Session isolation: Each user interaction runs in isolated environments
Extended execution: Support for both real-time interactions and long-running workloads
Built-in observability: CloudWatch integration, traces, and logs out of the box
Framework agnostic: Works with CrewAI, LangGraph, Strands Agents, Google ADK, OpenAI Agents, and custom frameworks
Consumption-based pricing: Pay only for what you use

En la práctica esto significa that you focus on building intelligent agents while AWS handles all the operational complexity.

The Architecture

Here's what we're building: a CrewAI multi-agent system deployed on AgentCore Runtime using the AgentCore CLI with CodeZip deployment (no containers needed).

Local Development          →    AgentCore Runtime (AWS)
┌─────────────────┐            ┌──────────────────────┐
│ crewai_agent.py  │  agentcore │ Serverless Runtime    │
│ @app.entrypoint  │  deploy   │ Session Isolation     │
│ localhost:8080   │ ────────► │ Auto-scaling          │
│ CrewAI + Bedrock │           │ CloudWatch Logs       │
└─────────────────┘            └──────────────────────┘

Step 1: Setting Up the Environment

The AgentCore CLI is an npm package — not pip. This was one of the key things I learned (the old bedrock-agentcore-starter-toolkit pip package is deprecated).

# Install AgentCore CLI (requires Node.js 20+)
npm install -g @aws/agentcore

# Verify
agentcore --version

# Install Python SDK
pip install bedrock-agentcore crewai

Step 2: The Agent Code — The `@app.entrypoint` Pattern

The core of AgentCore integration is the BedrockAgentCoreApp with the @app.entrypoint decorator. This is what makes your code work both locally and in the cloud.

One critical lesson learned: use lazy imports for CrewAI. CrewAI has heavy dependencies that take more than 30 seconds to import, which exceeds AgentCore's initialization timeout. Moving the imports inside the handler solves this.

from bedrock_agentcore.runtime import BedrockAgentCoreApp
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

app = BedrockAgentCoreApp()

@app.entrypoint
def crewai_handler(payload, context):
    """Main AgentCore entrypoint — lazy imports to avoid cold start timeout."""
    try:
        # Lazy imports: CrewAI is heavy, import inside handler
        from crewai import Agent, Crew, Process, Task, LLM
        from crewai.tools import BaseTool

        class KnowledgeSearchTool(BaseTool):
            name: str = "knowledge_search"
            description: str = "Search AWS documentation and technical knowledge"
            def _run(self, query: str) -> str:
                kb = {
                    "s3": "Object storage with 99.999999999% durability",
                    "lambda": "Serverless compute with automatic scaling",
                    "agentcore": "Secure serverless runtime for AI agents",
                }
                results = [f"{k.upper()}: {v}" for k, v in kb.items()
                          if any(w.lower() in v.lower() for w in query.split())]
                return "\n".join(results) or "No results found"

        user_input = payload.get("prompt", "How can I help you?")
        logger.info(f"Processing: {user_input}")

        llm = LLM(model="bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0", temperature=0.1)

        researcher = Agent(
            role="AWS Research Specialist",
            goal="Find comprehensive AWS service information",
            backstory="Expert researcher with deep AWS knowledge",
            tools=[KnowledgeSearchTool()],
            llm=llm, verbose=False,
        )

        crew = Crew(
            agents=[researcher],
            tasks=[Task(description=f"Research: {user_input}",
                       agent=researcher, expected_output="Research findings")],
            process=Process.sequential, verbose=False,
        )

        result = crew.kickoff()
        return {
            "result": result.raw,
            "status": "success",
            "session_id": context.session_id,
        }
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return {"result": f"Error: {str(e)}", "status": "error"}

if __name__ == "__main__":
    app.run()

A few things to note about the model choice: I'm using us.anthropic.claude-haiku-4-5-20251001-v1:0 — an inference profile ID, not a raw model ID. Newer Anthropic models on Bedrock require inference profiles (prefixed with us. or global.). Also, CrewAI uses assistant message prefill internally, which some newer Claude models don't support — Haiku 4.5 works perfectly.

Step 3: Local Testing

When you run your agent file, AgentCore automatically starts a web server on localhost:8080. No Flask, no FastAPI — it's all handled for you.

# Start the agent locally
python crewai_agent.py

# Test in another terminal
curl -X POST http://localhost:8080/invocations \
  -H "Content-Type: application/json" \
  -d '{"prompt": "What is AWS Lambda?"}'

Step 4: Deploy to AgentCore Runtime

This is where it gets real. The AgentCore CLI handles everything — project scaffolding, CDK infrastructure, and deployment.

# Create project (generates agentcore.json + CDK infrastructure)
agentcore create --name CrewAIAgent --defaults

# Copy your agent code into the project
cp crewai_agent.py CrewAIAgent/app/CrewAIAgent/main.py

# Configure deployment target (account + region)
# Edit CrewAIAgent/agentcore/aws-targets.json:
# [{"name": "default", "account": "YOUR_ACCOUNT_ID", "region": "us-east-1"}]

# Deploy (uses AWS CDK under the hood)
cd CrewAIAgent
agentcore deploy --yes

The deploy process:

Packages your code as a CodeZip archive
Uses AWS CDK to synthesize and provision CloudFormation resources
Creates IAM roles and AgentCore Runtime endpoint
Configures CloudWatch logging and observability

Important: The AgentCore execution role needs Marketplace permissions for third-party models (Anthropic). After deploy, add this to the role:

iam.put_role_policy(
    RoleName=role_name,
    PolicyName='MarketplaceModelAccess',
    PolicyDocument=json.dumps({
        'Version': '2012-10-17',
        'Statement': [{
            'Effect': 'Allow',
            'Action': ['aws-marketplace:ViewSubscriptions',
                      'aws-marketplace:Subscribe',
                      'aws-marketplace:Unsubscribe'],
            'Resource': '*'
        }]
    })
)

The notebook handles this automatically in the deploy cell.

Step 5: Invoke Your Deployed Agent

# Using AgentCore CLI
agentcore invoke --prompt "What is AWS Lambda? Answer in 3 sentences." --json

# Response:
# {
#   "response": "AWS Lambda is a serverless compute service that allows you to run
#    code without provisioning or managing servers, with automatic scaling capabilities
#    to handle varying workloads. Lambda executes code in response to events from
#    various AWS services and automatically scales your application by running code
#    only when needed. You pay only for the compute time you consume, making it a
#    cost-effective solution for event-driven applications and microservices architectures."
# }

You can also invoke programmatically with boto3:

import boto3, json, uuid

client = boto3.client('bedrock-agentcore')
response = client.invoke_agent_runtime(
    agentRuntimeArn="arn:aws:bedrock-agentcore:us-east-1:123456789012:runtime/your-runtime-id",
    runtimeSessionId=str(uuid.uuid4()),  # Required parameter
    payload=json.dumps({"prompt": "What is AWS Lambda?"}).encode(),
    qualifier="DEFAULT"
)

Step 6: Cleanup

agentcore remove all --yes
agentcore deploy --yes  # Deploys empty state to tear down resources

Lessons Learned

Building this end-to-end, I hit several gotchas that aren't obvious from the docs:

Issue	Solution
Old Starter Toolkit	Use `npm install -g @aws/agentcore`, not `pip install bedrock-agentcore-starter-toolkit`
Init timeout (30s)	Lazy imports — move `import crewai` inside the handler
Model prefill error	Use Claude Haiku 4.5 (supports assistant prefill that CrewAI needs)
Inference profiles	Use `us.anthropic.claude-*` prefix, not raw model IDs
Marketplace permissions	Add `aws-marketplace:ViewSubscriptions` and `Subscribe` to execution role
aws-targets.json	Field is `account` (not `accountId`)
AWS credentials for CLI	Export SSO credentials to env vars before running `agentcore deploy`

Key Takeaways

Lo interesante es que AgentCore eliminates the traditional deployment complexity for AI agents:

The AgentCore CLI (@aws/agentcore) is the current tool — it uses AWS CDK, not CodeBuild/ECR
CodeZip deployment means no Dockerfiles — just your Python code in a zip
@app.entrypoint is the only decorator you need to go from local to cloud
Lazy imports are essential for heavy frameworks like CrewAI
15 regions available for AgentCore Runtime
Consumption-based pricing — you pay only for actual compute time

Mi recomendación: start with the notebook, run it end-to-end, and then adapt the agent code for your use case. The deployment flow is surprisingly smooth once you know the gotchas.

Connect with me:

LinkedIn - Let's discuss AI and cloud architecture
X/Twitter - Follow for AWS and GenAI updates
GitHub - Check out my latest projects
Dev.to - More technical deep-dives
AWS Community - Join the conversation

I'm Carlos Cortez, this is Breaking the Cloud, and remember — the best way to learn is by building. See you in the next one!

Building Your First RAG System with Amazon Bedrock Agents and FAISS: A Developer's Journey

Carlos Cortez 🇵🇪 [AWS Hero] — Wed, 03 Sep 2025 03:14:15 +0000

Building Your First RAG System with Amazon Bedrock Agents and FAISS: A Developer's Journey

Discover how AI is transforming the way we build intelligent applications in the cloud. Let's explore how to create your first Retrieval-Augmented Generation (RAG) system using Amazon Bedrock Agents (managed AI agents service) with FAISS (Facebook AI Similarity Search) as our local vector database, perfect for those first steps into GenAI territory.

Why FAISS for Your First RAG Adventure?

The idea here is simple: when you're starting with RAG (Retrieval-Augmented Generation, a technique to enrich LLMs with external data), you want to focus on understanding the core concepts without getting lost in database configurations. FAISS runs locally, it's fast, and gives you complete control over your vector operations. Plus, it's free and doesn't require any AWS infrastructure setup for the vector storage part.

In practice, this means FAISS is your local playground where you can experiment, learn, and prototype before moving to production-ready solutions like Amazon OpenSearch Serverless or Amazon MemoryDB.

The Architecture We're Building

Our setup combines the best of both worlds:

Amazon Bedrock Agents: Handles the orchestration, reasoning, and LLM (Large Language Model) interactions
FAISS: Manages vector storage and similarity search locally
Custom Action Group: Bridges the agent with our FAISS operations

User Query → Bedrock Agent → Action Group → FAISS Search → Context → LLM Response

Setting Up Your Local FAISS Environment

Let's explore the initial configuration. What's interesting is that we can have everything running with just a few key components:

import boto3
import faiss
import numpy as np
from sentence_transformers import SentenceTransformer
import json
import uuid
from typing import List, Dict, Any

# Initialize the embedding model
embedding_model = SentenceTransformer('all-MiniLM-L6-v2')
embedding_dimension = 384

# Create FAISS index
index = faiss.IndexFlatIP(embedding_dimension)  # Inner Product for cosine similarity
document_store = {}  # Store original documents with metadata

Creating Your Document Ingestion Pipeline

The idea here is to create a simple but effective ingestion system. My recommendation is to start with this class that encapsulates all the functionality:

class FAISSDocumentStore:
    def __init__(self, embedding_model_name='all-MiniLM-L6-v2'):
        self.embedding_model = SentenceTransformer(embedding_model_name)
        self.dimension = self.embedding_model.get_sentence_embedding_dimension()
        self.index = faiss.IndexFlatIP(self.dimension)
        self.documents = {}

    def add_documents(self, texts: List[str], metadata: List[Dict] = None):
        """Add documents to the FAISS index"""
        if metadata is None:
            metadata = [{}] * len(texts)

        # Generate embeddings
        embeddings = self.embedding_model.encode(texts, normalize_embeddings=True)

        # Add to FAISS index
        start_id = len(self.documents)
        self.index.add(embeddings.astype('float32'))

        # Store documents with metadata
        for i, (text, meta) in enumerate(zip(texts, metadata)):
            doc_id = start_id + i
            self.documents[doc_id] = {
                'text': text,
                'metadata': meta,
                'id': doc_id
            }

    def search(self, query: str, k: int = 5) -> List[Dict]:
        """Search for similar documents"""
        query_embedding = self.embedding_model.encode([query], normalize_embeddings=True)
        scores, indices = self.index.search(query_embedding.astype('float32'), k)

        results = []
        for score, idx in zip(scores[0], indices[0]):
            if idx != -1:  # Valid result
                doc = self.documents[idx].copy()
                doc['similarity_score'] = float(score)
                results.append(doc)

        return results

# Initialize your document store
doc_store = FAISSDocumentStore()

Building the Bedrock Agent Action Group

Let's explore how to create the action group that will handle RAG operations. In practice, this means creating a Lambda function that acts as a bridge:

def lambda_handler(event, context):
    """
    Lambda function to handle Bedrock Agent action group calls
    """

    # Parse the incoming request
    action_group = event.get('actionGroup', '')
    api_path = event.get('apiPath', '')
    http_method = event.get('httpMethod', '')
    parameters = event.get('parameters', [])

    # Extract query parameter
    query = None
    for param in parameters:
        if param.get('name') == 'query':
            query = param.get('value')
            break

    if not query:
        return {
            'messageVersion': '1.0',
            'response': {
                'actionGroup': action_group,
                'apiPath': api_path,
                'httpMethod': http_method,
                'httpStatusCode': 400,
                'responseBody': {
                    'application/json': {
                        'body': json.dumps({'error': 'Query parameter is required'})
                    }
                }
            }
        }

    try:
        # Perform FAISS search
        results = doc_store.search(query, k=3)

        # Format results for the agent
        context_documents = []
        for result in results:
            context_documents.append({
                'content': result['text'],
                'score': result['similarity_score'],
                'metadata': result.get('metadata', {})
            })

        response_body = {
            'query': query,
            'documents': context_documents,
            'total_results': len(context_documents)
        }

        return {
            'messageVersion': '1.0',
            'response': {
                'actionGroup': action_group,
                'apiPath': api_path,
                'httpMethod': http_method,
                'httpStatusCode': 200,
                'responseBody': {
                    'application/json': {
                        'body': json.dumps(response_body)
                    }
                }
            }
        }

    except Exception as e:
        return {
            'messageVersion': '1.0',
            'response': {
                'actionGroup': action_group,
                'apiPath': api_path,
                'httpMethod': http_method,
                'httpStatusCode': 500,
                'responseBody': {
                    'application/json': {
                        'body': json.dumps({'error': str(e)})
                    }
                }
            }
        }

Configuring Your Bedrock Agent

What's interesting is that we can use AWS CLI to create the entire setup. My recommendation is to use this approach:

# Create the agent
aws bedrock-agent create-agent \
    --agent-name "faiss-rag-agent" \
    --description "RAG agent using FAISS for vector search" \
    --foundation-model "anthropic.claude-3-sonnet-20240229-v1:0" \
    --instruction "You are a helpful assistant that can search through documents to answer questions. When a user asks a question, use the search_documents function to find relevant information, then provide a comprehensive answer based on the retrieved context." \
    --region us-east-1

# Create action group (after deploying your Lambda function)
aws bedrock-agent create-agent-action-group \
    --agent-id "YOUR_AGENT_ID" \
    --agent-version "DRAFT" \
    --action-group-name "document-search" \
    --description "Search documents using FAISS vector database" \
    --action-group-executor lambda="arn:aws:lambda:us-east-1:YOUR_ACCOUNT:function:faiss-rag-function" \
    --api-schema '{
        "openapi": "3.0.0",
        "info": {
            "title": "Document Search API",
            "version": "1.0.0"
        },
        "paths": {
            "/search": {
                "post": {
                    "description": "Search for relevant documents",
                    "parameters": [
                        {
                            "name": "query",
                            "in": "query",
                            "required": true,
                            "schema": {
                                "type": "string"
                            },
                            "description": "The search query"
                        }
                    ]
                }
            }
        }
    }' \
    --region us-east-1

Loading Your First Documents

Let's explore how to populate our FAISS index with sample documents:

# Sample documents about AWS services
sample_docs = [
    "Amazon S3 is a highly scalable object storage service that offers industry-leading durability, availability, and performance.",
    "AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume.",
    "Amazon EC2 provides secure, resizable compute capacity in the cloud. It's designed to make web-scale cloud computing easier.",
    "Amazon RDS makes it easy to set up, operate, and scale a relational database in the cloud.",
    "Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale."
]

metadata = [
    {"service": "S3", "category": "Storage"},
    {"service": "Lambda", "category": "Compute"},
    {"service": "EC2", "category": "Compute"},
    {"service": "RDS", "category": "Database"},
    {"service": "DynamoDB", "category": "Database"}
]

# Add documents to FAISS
doc_store.add_documents(sample_docs, metadata)
print(f"Added {len(sample_docs)} documents to FAISS index")

Testing Your RAG System

The idea here is to validate that everything works before connecting the components. In practice, this means:

# Test the search functionality
test_query = "What database services does AWS offer?"
results = doc_store.search(test_query, k=2)

print(f"Query: {test_query}")
print("Results:")
for i, result in enumerate(results, 1):
    print(f"{i}. Score: {result['similarity_score']:.3f}")
    print(f"   Text: {result['text']}")
    print(f"   Service: {result['metadata'].get('service', 'N/A')}")
    print()

Connecting Everything with Bedrock Agents SDK

What's interesting is that we can interact with our agent programmatically:

import boto3

def chat_with_rag_agent(query: str, agent_id: str, agent_alias_id: str):
    """
    Chat with the Bedrock agent that uses FAISS for RAG
    """
    bedrock_agent_runtime = boto3.client('bedrock-agent-runtime', region_name='us-east-1')

    try:
        response = bedrock_agent_runtime.invoke_agent(
            agentId=agent_id,
            agentAliasId=agent_alias_id,
            sessionId=str(uuid.uuid4()),
            inputText=query
        )

        # Process the streaming response
        full_response = ""
        for event in response['completion']:
            if 'chunk' in event:
                chunk = event['chunk']
                if 'bytes' in chunk:
                    full_response += chunk['bytes'].decode('utf-8')

        return full_response

    except Exception as e:
        print(f"Error invoking agent: {str(e)}")
        return None

# Example usage
agent_response = chat_with_rag_agent(
    query="What are the benefits of using AWS Lambda?",
    agent_id="YOUR_AGENT_ID",
    agent_alias_id="YOUR_ALIAS_ID"
)

print("Agent Response:", agent_response)

Optimizing Your FAISS Performance

In practice, this means that when you scale up, consider these optimizations:

# For larger datasets, use IndexIVFFlat for faster search
def create_optimized_index(dimension: int, nlist: int = 100):
    """
    Create an optimized FAISS index for larger datasets
    """
    quantizer = faiss.IndexFlatIP(dimension)
    index = faiss.IndexIVFFlat(quantizer, dimension, nlist)
    return index

# Add GPU support if available
def create_gpu_index(dimension: int):
    """
    Create GPU-accelerated FAISS index
    """
    if faiss.get_num_gpus() > 0:
        res = faiss.StandardGpuResources()
        index = faiss.IndexFlatIP(dimension)
        gpu_index = faiss.index_cpu_to_gpu(res, 0, index)
        return gpu_index
    else:
        return faiss.IndexFlatIP(dimension)

Common Pitfalls and How to Avoid Them

1. Embedding Model Consistency
Always use the same embedding model for indexing and querying. Mixing models will give you poor results.

2. Normalization Matters

# Always normalize embeddings for cosine similarity
embeddings = model.encode(texts, normalize_embeddings=True)

3. Batch Processing
For large document sets, process in batches to avoid memory issues:

def add_documents_in_batches(doc_store, texts, batch_size=100):
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i+batch_size]
        doc_store.add_documents(batch)
        print(f"Processed batch {i//batch_size + 1}")

Next Steps and Production Considerations

This setup is perfect for learning and prototyping, but for production, my recommendation is:

Persistence: Save your FAISS index to disk

faiss.write_index(index, "my_index.faiss")

Scalability: Consider Amazon OpenSearch Serverless for production workloads
Monitoring: Add logging and metrics to your Lambda function
Security: Implement proper IAM roles and VPC configurations

El Takeaway Principal

The main takeaway is that starting with FAISS locally allows you to understand RAG fundamentals without additional complexities. Once you feel comfortable, you can explore Amazon Bedrock Knowledge Bases for a fully managed solution, or Amazon OpenSearch Serverless for more advanced vector search capabilities.

Lo interesante es que this approach allows you to iterate quickly, understand each component, and gradually move to more sophisticated setups as your needs grow.

Quick Actions for You

[ ] Set up your local Python environment with the required dependencies
[ ] Create a simple FAISS index with your own documents
[ ] Deploy the Lambda function for the action group
[ ] Configure your Bedrock Agent with the action group
[ ] Test the end-to-end RAG pipeline
[ ] Experiment with different embedding models
[ ] Try optimizing FAISS for your specific use case

My Final Recommendation

Start small with this local FAISS approach to understand RAG fundamentals. Once you're comfortable, explore Amazon Bedrock Knowledge Bases for a fully managed solution, or Amazon OpenSearch Serverless for more advanced vector search capabilities.

The beauty of this approach is that you can iterate quickly, understand every component, and gradually move to more sophisticated setups as your needs grow.

Thanks for reading, now it's your turn to try it out! Let's connect and share our experiences building with GenAI:

🔗 LinkedIn: https://www.linkedin.com/in/carloscortezcloud

🐦 X: https://x.com/ccortezb

💻 GitHub: https://github.com/ccortezb

📝 Dev.to: https://dev.to/ccortezb

🏆 AWS Heroes: https://builder.aws.com/community/@breakinthecloud

📖 Medium: https://ccortezb.medium.com

Stay curious, keep experimenting — and I'll catch you in the next one!

Starting with NLP on AWS from scratch — NLP Series part 1 | by Carlos Cortez | Breaking the Cloud

Carlos Cortez 🇵🇪 [AWS Hero] — Tue, 13 Apr 2021 01:17:56 +0000

Listen this article in our new mode: AudioBlog

It would be great to start segmenting our texts as well as the cover image, which is an Open Source software called Coreference created by Hugging Face, right? Let's go in parts ...

In case you are interested in hearing about Hugging Face and SageMaker, I spoke in depth a few days ago about this topic and its recent alliance with AWS here:

The learning curve

The learning curve is long and can get complex if we want to. For that there are flows and frameworks that will help us know where we are right now.

Natural Language Processing (NLP) flow *

Many times we skip key steps before we really understand each of these procedures. For example, data ingestion, web scraping, text analytics, EDA (exploratory analysis, and much more)

Remember CRISP-DM?

It is a framework for data mining, which is now known more as Machine Learning, Deep Learning, Data Science, etc.

Essentials to get started on AWS

Always to each place we go, we keep this pair in our pocket:

Python 3.x + boto3 *

** Boto3 *, is the AWS python library that will give us access to run and manage any resource within the cloud, as simple as * import boto3, ** and have your private keys configured or your roles correctly created .

Pre-trained models

Pre-trained models help us to shorten research, training and analysis time, and not start inventing what already exists. and for this we can use cloud services.

** Amazon Comprehend (Applied NLP) **, It is a SaaS type service, which means that it is ready to consume an API with the AWS boto3 library and begin to obtain insights from the data. It contains models already trained by Amazon Web Services, which we can reuse to start our own, and we will start knowing this great service and start building interesting applications using other AWS services.

** Hugging Face (advanced) **, later we will play with transformers, tokenizers, and more with these Deep Learning models of Hugging Face.

Build your own models:

Once we have a good understanding of NLP's self-managed cloud services, we can move to the next level with Amazon Sagemaker.

Or, with your own containers, which we will also learn, or locally

NLP Open Source Libraries

I will mention these for now and we will surely know more than 2 of this list in the coming months:

NLTK
Spark NLP
CoreNLP
SpaCy
Pytorch
Tensorflow
etc..

Tools and IDEs

From now on we will begin to know and learn step by step, what it is like to work in the cloud with an IDE for Data Science, and learn about AI services on AWS.

I recommend taking a look at the following, some paid and some free.

Sagemaker Studio / Sagemaker Notebooks
Jupyter Notebooks / Jupyter Lab
Databricks
Google Colab
among others ...

To start for free, I suggest ** Databricks and Colab **, the first one, has a community platform that allows you to create free clusters and they are eliminated if you do not use it for limited times, and Google's Colab, you can run it directly on your Google Drive, at no cost.

Deployment of my models in the cloud

It is not yet time to think about deploying, and yes, we will find many, many ways to do it, from the oldest to the most modern.

Traditional servers (VMs, Bare Metal)
Containers (such as AWS ECS, EKS, Fargate)
Without Servers (Functions as a Service)

This is a very good way to start, putting together serverless and Sagemaker.

Think Serverless First

Now we can start working without having to run servers using AWS Lambda, AWS Step Functions, Sagemaker Pipelines, AWS CodePipeline, which are services already deployed in the cloud that we can use to speed up our development and start-up.

In the following posts we learn how to deploy with AWS Lambda and Amazon Comprehend.

Amazon Comprehend and its superpowers

Can you imagine something as simple as writing a few lines of code to create a word detector and phrase extraction in just a few minutes instead of many hours of research? It is possible if you start to take advantage of Amazon Comprehend, a little Python and a couple of drops of Serverless

Phrase extraction:

We are using Python 3.6 here with boto3, the library

response **=** client**.**detect_key_phrases(

Text=’string’,

LanguageCode=’en’|’es’

)

Extraction of Entities, sentiment, language, syntax, topics, and document classification, are some of the additional things that we are going to do with Comprehend, as well as a specific version for Medicine, called ** Amazon Comprehend Medical **

What do we continue with?

Data ingestion and text Analytics is part of the important bases to understand NLP, so in the next post, we will start to build our first data ingestion of a web, to be able to obtain data, texts, titles, in a data frame to play.

** Next post: (on air on April 02) **

(Post Original: Medium and
Dev.cortez.cloud )

If you liked this post, give it a like, share and comment.

Learn by breaking things.

Subscribe to my channel, Breaking the Cloud and be Up to date with AWS at https://cortez.cloud

⭐Subscribe to my channel : http://bit.ly/aldiaconaws

videos, AWS news, analytics, reviews, demos, workshops

🔥🔥 Sígueme en mis redes 🔥🔥

follow me:

🦜 My Twitter: https://twitter.com/ccortezb

📺 Youtube Channel: http://bit.ly/aldiaconaws

📺 AWSUGPerú: https://www.youtube.com/awsusergroupperuoficial

📟 My Facebook: https://www.facebook.com/ccortezb/

🤳 My Instagram: ccortezbazan

📜My AWS Courses: https://cennticloud.thinkific.com

🕮 My blog — cortez.cloud

Thank you very much, I hope we see each other again

🔥🔥 About me 🔥🔥

Cortez.Cloud/whoami

Let me present to you, my little personal website https://www.Cortez.Cloud called “Breaking the Cloud”.

I will continue to create content each week from AWS on Al / ML, Serverless, Security and how to break the rules!

Also my next initiatives, workshops, courses, free videos, awsugperu and more.

Empezando con NLP en AWS desde cero - NLP Series 1

Carlos Cortez 🇵🇪 [AWS Hero] — Wed, 31 Mar 2021 04:15:22 +0000

Sería genial ya empezar a segmentar nuestros textos así como la imagen de portada que es un software Open Source llamado Coreference creado por Hugging Face, no? Vamos por partes…

Por si te interesa escuchar Sobre Hugging Face y SageMaker, hablé hace unos días profundamente sobre este tema y su reciente alianza con AWS aquí:

Al día con AWS - Ep 4 La semana de ML con algunas ROSAs en AWS (Marzo 21 al 29)

La curva de Aprendizaje

La curva de aprendizaje es larga y puede volverse compleja si quisiéramos. Para eso existen los flujos y los frameworks que nos van a ayudar a saber en qué parte estamos ahora mismo.

Flujo de Procesamiento de lenguaje natural (NLP)

Muchas veces saltamos pasos claves antes de entender realmente cada uno de estos procedimientos. Por ejemplo, la ingesta de datos, web scraping, text analytics, EDA (exploratory analysis, y mucho más)

¿Recuerdan CRISP-DM?

Es un framework para realizar minería de datos, lo que ahora se conoce más como Machine Learning, Deep Learning, Data Science, etc..

Lo indispensable para empezar en AWS
Siempre a cada lugar que vayamos, guardemos en el bolsillo esta dupla:

Python 3.x + boto3

Boto3, es la librería de python de AWS que nos dará acceso a ejecutar y administrar cualquier recurso dentro de la nube, tan sencillo como un import boto3, y tener tus llaves privadas configuradas o tus roles correctamente creados.

Modelos pre-entrenados

Los modelos pre entrenados nos ayudan a acortar tiempo de investigación, de entrenamiento y de análisis, y no empezar a inventar lo que ya existe.

Amazon Comprehend (NLP Aplicado), Es un servicio de tipo SaaS, que quiere decir que está listo para poder consumir un API con la librería de AWS boto3 y empezar a obtener insights de los datos. Contiene modelos ya entrenados por Amazon Web Services, que podemos re utilizar para empezar los nuestros, e iniciaremos conociendo este gran servicio y empezar a construir aplicaciones interesantes usando otros servicios de AWS.

Hugging Face (avanzado), más adelante jugaremos con transformers, tokenizers, y más con estos modelos de Deep Learning de Hugging Face.

Construir tus propios modelos:

Una vez entendamos muy bien los servicios auto administrados en la nube de NLP, podemos pasar al siguiente nivel ya se con Amazon Sagemaker
o con tus propios Containers, que también lo aprenderemos, o en modo local

Librerías NLP Open Source

Mencionaré estos por ahora y seguro conoceremos más de 2 de esta lista durante los próximos meses:

NLTK
Spark NLP
CoreNLP
SpaCy
Pytorch
Tensorflow etc..

Tools e IDEs

Desde ahora empezaremos a conocer y aprender paso a paso, cómo es trabajar en la nube con una IDE para Ciencia de Datos, y conocer los servicios de AI en AWS.

Recomiendo echar un vistazo a las siguientes, algunas de pago y otras gratuitas.

Sagemaker Studio / Sagemaker Notebooks
Jupyter Notebooks / Jupyter Lab
Databricks
Colab de Google entre otros...

Para empezar de manera gratuita, sugiero Databricks y Colab, el primero, tiene una plataforma de comunidad que te permite crear cluster gratuitos y se eliminan si no lo usas durante tiempos limitados, y Colab de Google, puedes ejecutarlo directo en tu Google Drive, sin costo alguno.

Despliegue de mis modelos en la nube

Aún no es momento de pensar en desplegar, y sí, encontraremos muchísimas maneras de hacerlo, desde la más antigua, a la más moderna.

Servidores tradicionales (VMs, Bare Metal)
Contenedores (como AWS ECS, EKS, Fargate)
Sin Servers (Funciones como Servicio)

Esta es una muy buena manera de empezar, juntando serverless y Sagemaker.

Think Serverless First

Ahora podemos empezar a trabajar sin tener que correr servidores usando AWS Lambda, AWS Step Functions, Sagemaker Pipelines, AWS CodePipeline, que son servicios ya desplegados en la nube que podemos utilizar para agilizar nuestro desarrollo y puesta en marcha.

En los siguientes posts aprendemos a desplegar con AWS Lambda y Amazon Comprehend.

Amazon Comprehend y sus superpoderes

Imaginas algo tan sencillo como escribir algunas líneas de código para crear un detector de palabras y extracción de frases en tan solo unos minutos en vez de muchas horas de investigación? Es posible si empiezas a sacarle provecho a Amazon Comprehend, un poco de Python y un par de gotitas de Serverless

Extracción de frases:

Estamos usando aquí Python 3.6 con boto3, la librería

response = client.detect_key_phrases(
    Text='string',
    LanguageCode='en'|'es'
)

Extracción de Entidades, sentimiento, lenguaje, sintaxis, tópicos, y clasificación de documentos, son algunas de las cosas adicionales que vamos a realizar con Comprehend, así como una versión específica para Medicina, llamada Amazon Comprehend Medical

¿Con qué continuamos?

La ingesta de datos y el text Analytics, es parte de las bases importantes para comprender NLP, así que en el siguiente post, empezaremos a construir nuestra primera ingesta de datos de una web, para poder obtener datos, textos, títulos, en un data frame para jugar.

Siguiente post: (Al Aire el 02 de Abril)

Si te gustó este post, dale un like, comparte y comenta.

Rompiendo se aprende

Suscríbete a mi canal, Breaking the Cloud y Al día con AWS en https://cortez.cloud

⭐Suscríbete a mi canal : http://bit.ly/aldiaconaws

videos, noticas de AWS, análisis, demos, workshops

🔥🔥 Sígueme en mis redes 🔥🔥

follow <- me()

🦜 Mi Twitter: https://twitter.com/ccortezb

📺 Youtube Channel: http://bit.ly/aldiaconaws

📺 AWSUGPerú: https://www.youtube.com/awsusergroupperuoficial

📟 Mi Facebook: https://www.facebook.com/ccortezb/

🤳 Mi Instagram: ccortezbazan

📜 Mis cursos de AWS: https://cennticloud.thinkific.com

🕮 Mi blog — cortez.cloud

Muchas gracias, espero nos volvamos a ver

🔥🔥 Acerca de mí 🔥🔥

Cortez.Cloud/whoami

Les presento mi pequeña web personal https://www.Cortez.Cloud llamado “Breaking the Cloud”.

Seguiré creando contenido cada semana de AWS sobre Al/ML, Serverless, Security y como romper las reglas!

También mis próximas iniciativas, talleres, cursos, videos gratuitos, awsugperu y más.

Trabajando con Amazon DocumentDB desde mi local

Carlos Cortez 🇵🇪 [AWS Hero] — Tue, 30 Mar 2021 17:46:02 +0000

Hoy vamos a continuar con la 2da parte de las series de Aprendiendo DocumentDB en AWS, Si aún no has visto la 1ra parte, ve aquí: https://www.cortez.cloud/blog/primeros-pasos-con-amazon-documentdb-y-aws-cli

Siempre es importante trabajar localmente, y casi siempre se cometen errores y descuidos que hacen que nuestros ambientes de desarrollo estén expuestos a conexiones locales inseguras. Para ello vamos a asegurar nuestra conexión poco a poco, y en este post aprenderemos a usar las herramientas que nos permitirán una mejor administración de nuestra instancia.

En el anterior post, explicamos cómo crear un cluster e instancias de DocumentDB con AWS CLI, así que partamos desde allí y veamos cómo trabajar localmente con ello.

Usando Robo3T (Ex Robomongo)

Primero que todo veamos la arquitectura esencial y más eficiente para trabajar con DocumentDB:

Me conecto desde mi escritorio y voy instalando Robo3T y mongodb client 3.6 (funciona para mac, windows, linux)

https://robomongo.org/download

Bonita arquitectura, pero ¿Qué es un Bastion Host?

Es un servidor puente, en el cual usaremos recursos para poder saltar a otras partes de la red. En este caso, lo usaremos para saltar de una red pública a una privada que es dónde estará docdb.

Ejemplo:
un EC2, una máquina virtual, una droplet, etc

Veamos paso a paso, cómo configurar desde cero mi conexión hacia mi cluster

Crear un ec2 como bastion Host:

Ya hemos visto cómo usar la CLI, así que hagamos lo siguiente y creemos un EC2, Security groups y agreguemos permisos para poder acceder desde la IP que tengas en casa:

$ aws ec2 create-security-group --group-name docdb --description "docdb" --vpc-id  vpc-81a06de4
{
    "GroupId": "sg-031f86d07f253eacf"
}

Ahora, verifica tu IP Pública, también por cli, somos unos fanáticos de la cli ahora!

curl https://checkip.amazonaws.com
190.236.xxx.xxx

*(usa tu propio resultado para las siguientes líneas..)

Agreguemos estas reglas en nuestro nuevo SG para luego poder usarlo en el bastion host EC2:

Abrir puerto 22

aws ec2 authorize-security-group-ingress --group-id sg-031f86d07f253eacf --protocol tcp --port 22 --cidr 190.236.xxx.xxx

Abrir puerto 27017 y 27018 (mongodb y adicional para el bastion host)

aws ec2 authorize-security-group-ingress --group-id sg-031f86d07f253eacf --protocol tcp --port 27017 --cidr 190.236.xxx.xxx

aws ec2 authorize-security-group-ingress --group-id sg-031f86d07f253eacf --protocol tcp --port 27018 --cidr 190.236.xxx.xxx

habilitar acceso de mongodb hacia sí mismo:

aws ec2 authorize-security-group-ingress --group-id sg-031f86d07f253eacf --protocol tcp --port 27017 --source-group
 sg-031f86d07f253eacf

Creamos ahora la instancia:

$ aws ec2 run-instances --image-id ami-067f5c3d5a99edc80 --count 1 --instance-type t2.micro --key-name ~/llaves/docdb-ccortez.pem --security-group-ids sg-031f86d07f253eacf --subnet-id subnet-6d558c08

Configuramos SSH tunnel para poder habilitar o deshabilitar cuando queramos:

sudo vim .ssh/config

Host tunnel
HostName 52.89.199.68
User ec2-user
IdentitiesOnly yes
IdentityFile ~/llaves/docdb-ccortez.pem
LocalForward 27018 docdb-001.cluster-cgyttffk8nhp.us-west-2.docdb.amazonaws.com:27017:27017

También es posible crear un bashero ejecutable aparte para no sobreescribir el archivo ~/.ssh/config

OJO: puedes refrescar, borrar caché o abrir una terminal nueva para renovar la conexión del túnel en caso tengas problemas para abrirlo, o te aparezca algún error.

Iniciamos nuestra conexión con Robo3T, así, en la 1ra pestaña,

Config inicial:

Autenticación

Opcional:
Database: admin (no vamos a poder ver la base de datos admin pero si todas las demás)

SSH tunnel:

SSL Certificate (en caso sea):

(Advanced Options)
Invalid Hostnames: Allowed

Test final de conexión:

En caso sea un cluster creado sin SSL: (opcional, en este ejemplo usé otro cluster que creé sin SSH llamado docdb2)

Conclusiones:

Conectarse a Docdb es fácil, y seguro.
Nos incentiva a crear arquitecturas más seguras, a usar más la capa privada para las bases de datos.
Con estos tutoriales, irás aprendiendo a usar una base de datos NoSQL desde cero.

Próximos pasos

En el próximo post, empezaremos con las migraciones hacia Amazon DocumentDB con AWS DMS.

Aún necesitamos ahondar más en la seguridad y los accesos, así como seguir aprendiendo más de AWS CLI.

Suscríbete a mi canal, Breaking the Cloud y Al día con AWS en https://cortez.cloud
Si te gustó este post, dale un like, comparte y comenta.
⭐Suscríbete a mi canal : http://bit.ly/aldiaconaws

🔥🔥 Sígueme en mis redes 🔥🔥

🦜 Mi Twitter: https://twitter.com/ccortezb

📺 Youtube Channel: Al día con AWS / Breaking The Cloud: http://bit.ly/aldiaconaws

📺 AWS Perú Community AWS UG Perú Oficial: https://www.youtube.com/awsusergroupperuoficial

📟 Mi Facebook: https://www.facebook.com/ccortezb/

🤳 Mi Instagram: ccortezbazan

📜 Mis cursos de AWS: https://cennticloud.thinkific.com

🕮 Mi blog — https://cortez.cloud

🔥🔥 Acerca de mí 🔥🔥

https://www.cortez.cloud/whoami

Les presento mi pequeña web personal https://www.Cortez.Cloud llamado “Breaking the Cloud”.

Seguiré creando contenido cada semana de AWS sobre Al/ML, Serverless, Security y como romper las reglas!

También mis próximas iniciativas, talleres, cursos, videos gratuitos, awsugperu y más.

aws #breakingthecloud

Análisis en caliente de la Keynote de Andy Jassy Parte 1 - reinvent 2020

Carlos Cortez 🇵🇪 [AWS Hero] — Wed, 02 Dec 2020 01:40:08 +0000

Hola a todos, bienvenido a #BreakingtheCloud, yo soy Carlos Cortez @ccortezb y esto es cobertura del re:invent 2020 día tras día.

¡Vayamos directo al grano!

Nuevas instancias y más nombres que aprenderse

Alguna de ellas que soportan Arm Graviton2
Amazon EC2 C6gn! Con 100 gbps

Durante la mañana también fue lanzado:
*D3dn (Dense Storage)
*R5b (EBS Performance)
*M5zn (ntel Xeon Scalable CPU)
*G4ad (AMD GPUs)

Nuevas características:

Nueva EC2 D3 y D3dn hace referencia a instancias con bajo costo en discos magnéticos. Son perfectos para procesamiento big data y montar los datos en HDFS, procesamiento de logs, etc.

Cabe resaltar que el costo de Terabytes es 80% menor que cuando usábamos D2.

Nueva EC2 R5b, nueva en la familia, con AWS Nitro System, con 60 gbps de ancho de banda en el EBS y con 260 mil de IOPS ya se nos desborda de las manos! OMG! La R5 norma solo llega hasta 19 gbps

Cuando pensábamos que ya habíamos tocado el cielo, viene la nueva EC2 M5zn! La evolución de la z1d, también con AWS Nitro System y EFA (Elastic Fabric Adapter) especialmente diseñada para HPC.

Pero, ¿para qué se necesitaría este tipo de instancia? Lo vimos en la Keynote, un buen ejemplo es BOOM, Supersonic Passenger Airplanes.

*Industria Aeroespacial
*Automoviles
*Aplicaciones financieras
*Simulación de modelamiento de aplicaciones
*Energía, etc..

Habana Gaudi-based Amazon EC2 instances

La verdadera competencia de NVIDIA

AWS, nunca escapa poder presentar nueva tecnología para machine learning, pero que cosa realmente es Habana Gaudi?
Habana Labs es una start-up israelí, adquirida luego por Intel que tienen los Procesadores para AI, llamados Gaudi AI processors.

Excelentes para correr modelos de deep learning, en todas sus gamas: NLP, Computer vision, motor de recomendaciones, etc.
Por qué Habana Gaudi?

40% más económicas que las anteriores EC2 con procesamiento de GPU para ML. Y Lo más impactante es que tendremos hasta 8 tarjetas Gaudi:

An 8-card Gaudi EC2 instance can process about 12,000 images->per-second training the ResNet-50 model on TensorFlow. Each >Gaudi processor integrates 32GB of HBM2 memory and features >RoCE on-chip integration used for inter-processor >connectivity inside the server. Scaling across servers will >be enabled using the AWS Elastic Fabric Adapter (EFA) >technology, allowing AWS and its customers to seamlessly >expand use of multiple Gaudi based systems for efficient and >scalable distributed training.

Fuente: Hanaba: https://habana.ai/habana-gaudi-ai-processors-to-bring-lower-cost-to-train-to-amazon-ec2-customers/

El hermano menor de Inferencia: AWS Trainium

Un Billón de cálculos por segundo? La mayor cantidad de TFLOPS, chip de entrenamiento? ¿Qué son estas cosas y cómo lo entendemos más fácil?

Primero que todo, volvamos a Inferentia, que fue lanzado en el 2019. Palabras claves?

Chip de inferencia para modelos de ML
Neuron SDK, ¿qué es? Un compilador.
Instancias Inf1 EC2, que corren con el chip de Inferencia.

Aprendamos los términos:

El término FLOPS (Floating point operations per second) >significa “operaciones de coma flotante por segundo”, es una >unidad que se utiliza para medir cálculos matemáticos que >puede hacer por segundo una CPU o en una GPU.
De allí viene GigaFlops, es decir millones de cálculos por >segundo y TeraFlops, Billones de cálculos por segundo.

AWS Trainium sale a cubrir todo lo que Inferentia no hace, es decir la parte del entrenamiento. También funcionará con Neuron SDK y en instancias Inf1

Ya puedo usarlo?

No, aún estará disponible en los primeros meses del 2021. Así que atento a las noticias.

Nos alineamos con las nubes. ECS y EKS desde cualquier lado.

Ahora podremos correr cargas de trabajo literalmente desde donde se nos plazca,

Como funcionará?

Seguiremos usando el Clásico ECS Control Plane dentro de la región
Se instalará el agente de AWS Systems Manager
Se agrega y veremos un nuevo tipo de opción para lanzar ECS, aparte de los ya conocidos “EC2” y “FARGATE” se tendrá uno nuevo llamado “EXTERNAL”.

Revisen el blog oficial donde encontrarán una demo y un diagrama de cómo se verá realmente:

EKS Distro de código abierto!

ECS no es el único que recibe una actualización, sino que EKS también podrá “desplegarse en cualquier lado” de una manera distinta. La distribución de Kubernetes que usa EKS será puesta a disposición de la comunidad.

Veamos qué podemos construir con esta nueva característica en la que recién tendremos acceso a mediados del 2021.

Github Repo:

https://github.com/aws/eks-distro

Lambda ahora nos cobra por cada 1 ms. Debo preocuparme?

Serverless Lovers, acaso no es genial que ahora Lambda nos cobre por cada milisegundo procesado de cómputo?

Solo si tienen cargas de trabajo de millones y millones de transacciones verás la diferencia sustancial.
Tengo que resaltar que también va a ser soportado por Provisioned Concurrency para realizar pruebas de carga. Esto sí está espectacular!

Ejemplo:

50 millones de transacciones de 100 ms vs 50 ms tiene una mejora de 25% en costos.

Desarrollando microservicios ahora con AWS Proton!, es en serio? Más vendor lock-in? más simplicidad?

Después de leerme parte de la documentación, tengo que decirles que aprovechen este post, porque les explicaré algo que me tomó varios minutos entenderlo. Vamos AWS, sí que hiciste algo complicado al inicio con todos esos componentes nuevos, templates, environments, services, services templates y más! Ufff, ahora sí vayamos a AWS Proton:

Qué realmente es?

Automatiza:

aprovisionamiento y
despliegue: Sólo para infraestructura relacionada con Serverless y Contenedores. Con 2 caminos: Equipo de infra aprovisiona y Equipo de desarrollo consume.
Infra estandarizada: CloudFormation
CI/CD Integrado para que sea consumido por los desarrolladores.

Cómo va funcionar?

Antes de nada según la documentación nos piden algunos pre-requisitos!!!!!
Permisos de Admin, IAM Roles, Saber CloudFormation y Jinja y pues un repositorio Github

Proton desde cero:

Creamos la plantilla de ambientes (environment template)
Creamos un ambiente (Proton Environment)
Crear una plantilla de servicio
Crear un Proton Service y desplegar una aplicación

Proton simplifica la entrega de infraestructura como también define un estándar. Es cierto que seguimos usando CloudFormation, posiblemente veamos integraciones internas con SAM y luego con third-party apps como Terraform que puedan ir agregándole a esta gran herramienta.

Genera vendor lock-in de alguna manera, seguimos usando CloudFormation, tal vez en el roadmap de desarrollo estén pensando integrar herramientas de testing para plantillas así como pruebas de seguridad,

Beneficios?

*Rapidez en la entrega de Infraestructura para los que ya *tenemos todo corriendo con CloudFormation.
*Simplicidad para el desarrollador elegir y desplegar en ambientes aislados
*Pruebas rápidas de aplicaciones

Estemos atentos a lo que se viene con AWS Proton!

¡Actualizados al fin! ¡Llegó la nueva generación de EBS! gp3 e io2

¿Qué hay de nuevo con gp3? 20% más económico y 4x mejor performance

Simplemente una muy buena opción, ahora tendremos 3,000 IOPS de base para todo el volumen, mientras que en gp2 tenemos 3 IOPS por GB.

Habrá que afinar más la calculadora para ver cuál conviene y en qué momento.

Acciones inmediatas: Migremos todo a gp3 right now!

Pueden revisar más en el blog oficial de AWS:
https://aws.amazon.com/es/blogs/aws/new-amazon-ebs-gp3-volume-lets-you-provision-performance-separate-from-capacity-and-offers-20-lower-price/

Y por último, tenemos a:

Nuevo EBS io2 Block Express al mismo precio que io1?

La nueva generación de los discos EBS especializados para IOPS, tienen un gran aumento en el ratio de IOPS por GB, subiendo a 500 IOPS por GB.

Lo interesante y raro al mismo tiempo es que está al mismo precio que su generación anterior y sport muchisimo más IOPS, de manera que es mandatorio que migremos todo lo que actualmente esté en io1 hacia io2

Algo tan simple como esto:

aws ec2 modify-volume --volume-id vol-0b3c663aeca5aabb7 --volume-type io2

Pueden revisar más en el blog oficial de AWS:

https://aws.amazon.com/es/blogs/aws/new-ebs-volume-type-io2-more-iops-gib-higher-durability/

Definitivamente fue un día muy largo y lleno de emociones. Lo vivimos tal cual fuese presencial, como un partido de Fútbol, cómo cuándo Perú volvió al mundial después de 36 años. Gritando cada nuevo servicio lanzado, especialmente los de Sagemaker que son mis favoritos y haré un post especial para cada uno de ellos!

Por ahora terminamos con esta parte 1 del Keynote, realmente espectacular y esperando probar estos servicios en estos días, espero les haya servido y por favor comenten y compartan.

En la siguiente parte:

Parte 1: EC2 y microservicios
Parte 2: Bases de datos y ML
Parte 3: Contact Center e Industrial IoT

Unanse al Slack de AWS UG Perú para recibir noticias en vivo del re:invent 2020,

Estén atentos para cuando lancemos los After Party de los anuncios en las noches, y puedas unirte a discutir con nosotros. Vive las reacciones en vivo, repasemos los nuevos servicios y más sorpresas y sorteos!

Slack: https://join.slack.com/t/aws-peru/shared_invite/zt-jt2324io-3VBHUQ8iH5rEkqGchKicEw

Carlos Cortez - AWS UG Perú Leader / AWS ML Community Builder
ccortez@aws.pe
@ccortezb
Podcast: imperiocloud.com @imperiocloud
twitch.tv/awsugperu
cennticloud.thinkific.com

Primeros pasos con Amazon DocumentDB y AWS CLI

Carlos Cortez 🇵🇪 [AWS Hero] — Tue, 20 Oct 2020 06:22:26 +0000

Hola a todos, bienvenido a #BreakingtheCloud, yo soy Carlos Cortez @ccortezb
Y estas serán unas series donde hablaremos de una de las bases de datos más interesantes en AWS: Amazon DocumentDB

Primero que todo debemos de saber de qué tipo de base de datos estamos hablando. DocumentDB, pertenece al grupo de bases de datos NoSQL orientadas a documentos.

NoSQL, significa Not Only SQL, donde los modelos de datos pueden ser esquemas flexibles que se adaptan a los requisitos de las aplicaciones más modernas.

Les recomiendo leer a Martin Fowler,

https://martinfowler.com/tags/noSQL.html

Pero, porqué deberíamos usar una base de datos orientada a documentos? Cuál es la verdadera ventaja? Qué tipos de aplicaciones son las que usualmente se desarrollan usando una base de datos orientada a documentos?

Vayamos paso a paso, no nos desesperemos, hay mucho por describir en el mundo de Nosql y más aún en Amazon Web Services

Lo primero que debemos saber, es que una de las bases de datos más conocidas y comerciales orientadas a documentos es MongoDB

Ahora, DocumentDB, según la documentación de AWS, DocumentDB tiene compatibilidad con MongoDB 3.6 exactamente en el momento que estoy escribiendo este post, pero qué significa que sea “compatible” realmente? La verdad detrás de este espectacular motor de base de datos es que no replica nada de MongoDB 3.6, sino que emula el comportamiento del motor, y de sus componentes.

Muchas de sus funcionalidades que empezaremos a conocer se pueden realizar en DocumentDB, la manera cómo hacemos las consultas, creación de índices también son compatibles y cómo usamos los comandos, se mantiene:

https://docs.aws.amazon.com/documentdb/latest/developerguide/what-is.html

Ahora que sabemos, revisemos los componentes y funcionalidades que tiene DocumentDB. Este servicio se compone por una capa de cluster, instancia y almacenamiento. Así como en Mongodb al crear múltiples instancias introducimos el término de replica set o ReplSet, en dónde existe un nodo máster y nodos réplicas. En DocumentDB las réplicas se conocen como nodos de lectura y el máster como nodo de escritura.

Manos a la obra ahora, empezamos a jugar con la línea de comandos de AWS, la famosa CLI, y lo haremos dentro de una instancia de Cloud9 con un InstanceRole

Primero creemos una instancia de Cloud9

Vamos a crear un instance Role, vamos al servicio IAM y le damos a create role y también agregamos documentDB Full Access al rol.

Elegimos las políticas de IAM necesarias, en este caso accesos Full para DocumentDB, esto nos permitirá acceder por líneas de comandos a todas la API de DocumentDB

Seleccionamos EC2 para que asuma el rol

Creamos un Instance Role en IAM y cambiamos en EC2 del Cloud9, al nuevo rol.

Luego solo reemplazamos el rol por defecto de Cloud9 por el nuevo creado. Otra opción es también solo editar el rol por defecto y agregarle las políticas que creas necesario.

Ahora toca desactivar las credenciales temporales en cloud9 para poder trabajar con Roles
Ir a configuracion
AWS Settings
Desactivar Temporary Credentials

SI quieres saber más de el principio de “Least privilege” dale un vistazo a IAM Keeper:
https://medium.com/@iamkeeper/achieving-least-privilege-permissions-in-aws-97ab8378fb2#:~:text=One%20of%20the%20basic%20principles,required%20to%20get%20job%20done.

Ahora que ya podemos ejecutar CLI en cloud9, vayamos a crear recursos de DocumentDB,

Ejemplo:

aws docdb create-db-cluster \
      --db-cluster-identifier docdb-cluster \
      --engine docdb \
      --deletion-protection \
      --master-username ccortez \
      --master-user-password cortez,,123 \
      --region us-west-2

Puedes ir verificando lo que creas con describe-db-clusters así:

Describe clusters:

aws docdb describe-db-clusters --filter Name=engine,Values=docdb --region us-west-2

Otra manera de buscar con filtros con la CLI es la siguiente:

Describe Clusters con filtros:

aws docdb describe-db-clusters \
   --filter Name=engine,Values=docdb \
   --db-cluster-identifier docdb-cluster \
   --region us-west-2

Ahora que el cluster está creado, vamos a crear instancias dentro del cluster:

Crear instancia de documentDB en el Cluster creado

aws docdb create-db-instance \
    --db-cluster-identifier docdb-cluster \
    --db-instance-class db.t3.medium \
    --db-instance-identifier docdb-cluster-instance01 \
    --engine docdb

(Opcional) Otra manera de lanzar clusters e instancias de DocumentDB es usar CloudFormation, la herramienta de

Infraestructura como código de AWS:

https://s3.amazonaws.com/amazon-documentdb-samples/v1/cloudformation-templates/documentdb_full_stack.yaml

Finalmente, necesitamos instalar el cliente de Mongodb 3.6 en nuestra consola de Cloud9,
Eso lo podemos hacer directamente con esta documentación oficial:

https://docs.mongodb.com/v3.6/tutorial/install-mongodb-on-amazon/

E intentemos conectarnos desde nuestra instancia Cloud9,

mongo --host docdb-cluster.cluster-cgyttffk8nhp.us-west-2.docdb.amazonaws.com:27017 --username ccortez --password ccortez,,123

Por ahora terminamos con este primer tutorial, espero les haya servido y por favor comenten y compartan.

Estas series de DocumentDB cuentan con los siguientes post:

Primeros pasos con Amazon DocumentDB y AWS CLI
Trabajando con Amazon DocumentDB desde mi local
Migrando hacia Amazon DocumentDB con AWS DMS
Migrando hacia Amazon DocumentDB de la manera tradicional
Configurando SSL en Amazon DocumentDB
Backup de Indices con Amazon DocumentDB Index Tool
Creación de perfiles de consultas de ejecución lenta en
Amazon DocumentDB (slow queries profiling)

Muchas gracias,

Carlos Cortez
ccortez@aws.pe
@ccortezb
Podcast: imperiocloud.com @imperiocloud
twitch.tv/breakingthecloud
cennticloud.thinkific.com

DEV Community: Carlos Cortez 🇵🇪 [AWS Hero]

I Built a Minecraft Mod Where Every Sword is an AWS Service — Here's How We Coded It with AI

I Built a Minecraft Mod Where Every Sword is an AWS Service — Here's How We Coded It with AI

Why AWS Swords?

The Architecture

The Tech Stack

Step 1: The Foundation — SwordAbility Interface

Step 2: BaseSword — The Shared Infrastructure

Step 3: Sword of Lambda — Serverless Allies

The LambdaMinionManager — Lifecycle Management

Step 4: Greatsword of Lambda — Mass Invoke

Step 5: Sword of S3 — Object Storage in Your Inventory

Step 6: Sword of EC2 — Auto Scaling Damage

Step 7: Animated Textures and Visual Polish

Step 8: Client-Side Particles

Step 9: Registration and Localization

Building and Running

Coding with AI — How We Actually Built This

Lessons Learned

What's Next

The Main Takeaway

How I Deployed CrewAI Multi-Agent Teams on Amazon Bedrock AgentCore Without Writing a Single Dockerfile

How I Deployed CrewAI Multi-Agent Teams on Amazon Bedrock AgentCore Without Writing a Single Dockerfile

Why AgentCore Changes the Game

The Architecture

Step 1: Setting Up the Environment

Step 2: The Agent Code — The @app.entrypoint Pattern

Step 3: Local Testing

Step 4: Deploy to AgentCore Runtime

Step 5: Invoke Your Deployed Agent

Step 6: Cleanup

Lessons Learned

Key Takeaways

Building Your First RAG System with Amazon Bedrock Agents and FAISS: A Developer's Journey

Building Your First RAG System with Amazon Bedrock Agents and FAISS: A Developer's Journey

Why FAISS for Your First RAG Adventure?

The Architecture We're Building

Setting Up Your Local FAISS Environment

Creating Your Document Ingestion Pipeline

Building the Bedrock Agent Action Group

Configuring Your Bedrock Agent

Loading Your First Documents

Testing Your RAG System

Connecting Everything with Bedrock Agents SDK

Optimizing Your FAISS Performance

Common Pitfalls and How to Avoid Them

Next Steps and Production Considerations

El Takeaway Principal

Quick Actions for You

My Final Recommendation

Starting with NLP on AWS from scratch — NLP Series part 1 | by Carlos Cortez | Breaking the Cloud

Listen this article in our new mode: AudioBlog

The learning curve

Remember CRISP-DM?

Essentials to get started on AWS

Pre-trained models

Build your own models:

NLP Open Source Libraries

Tools and IDEs

Deployment of my models in the cloud

Think Serverless First

Amazon Comprehend and its superpowers

What do we continue with?

Learn by breaking things.

🔥🔥 Sígueme en mis redes 🔥🔥

🔥🔥 About me 🔥🔥

Empezando con NLP en AWS desde cero - NLP Series 1

La curva de Aprendizaje

¿Recuerdan CRISP-DM?

Python 3.x + boto3

Modelos pre-entrenados

Construir tus propios modelos:

Librerías NLP Open Source

Tools e IDEs

Despliegue de mis modelos en la nube

Think Serverless First

Amazon Comprehend y sus superpoderes

Extracción de frases:

¿Con qué continuamos?

Rompiendo se aprende

Step 2: The Agent Code — The `@app.entrypoint` Pattern