DEV Community: N Chandra Prakash Reddy

From Panic to Peace: Mastering Zero-Downtime ECS Deployments

N Chandra Prakash Reddy — Wed, 19 Aug 2026 16:28:37 +0000

Friday afternoon. The code is merged, the pipeline is green and the release is ready. But instead of pressing the deploy button and heading out for the weekend, your team is haggling who needs to stick around "just in case" everything catches fire.

Let’s be real. Deploying new code to production is like placing a bet on the roulette table.

You start the app, watch the server logs like a hawk, and hope the customer service channel is silent. If an alert is triggered, there is panic. You hustle to determine what broke, patch it live, or execute a nasty manual rollback while users are down.

Does this sound familiar? This is a stressful routine and is very typical in software engineering. But it doesn’t have to be this way. Modern traffic-shifting tactics on Amazon Elastic Container Service (ECS) can turn stressful releases into repetitive, automated processes.

Now you can take control of your deployments with Blue-Green and Canary tactics.

The Vicious Cycle of Deployment Dread

Here is the point … when delivering software is hard, teams naturally want to do it less often.

If releasing code takes downtime, late night coordination and significant risk, you start batching your changes. You don't deploy a single minor feature on a Tuesday, you combine three weeks of updates into one enormous weekend release.

And that’s a huge problem. Larger batches of code have more variables, more possible conflicts and a much greater danger of breaking something vital. The pain when that huge discharge inevitably bombs strengthens your fear of deploying.

To break this loop you need to be able to test in a true production environment without impacting the users and you also need to be able to roll back changes immediately if something goes wrong.

The Two-House Strategy: Blue-Green Deployments

Imagine that you are moving into a new residence. You simply build an exact replica of your house next door instead of loading all your stuff into a truck, selling your old house and praying the new one has functional plumbing.

You bring your furniture in, check the water pressure, sleep on the bed for a night, and make sure it’s perfect. Then when you are happy you just move your mailing address to the new house. If the roof starts leaking the next day, you just change your address back and move next door to your previous, perfectly functional home.

This is called a Blue-Green deployment in the cloud world.

How the Dual Environment Works

You have the same infrastructural settings. We can call the existing live environment “Blue” and the freshly updated environment “Green”.

Step 1: The Shadow Launch. Your ECS pipeline distributes the new container image to the Green environment. At this stage 100% of your live client traffic is still going to the Blue environment.
Step 2: Private Validation. Your team does testing against the green environment. It is a totally accurate testing ground because it is connected to the same production databases and employs the same networking rules.
Step 3: The Swap. When the Green environment has passed all of the checks, you adjust your Application Load Balancer (ALB) to send traffic to the Green environment.
Step 4: The Safety Net. The old Blue environment is still running for a pre-determined cool down period.

You’re probably wondering… what if a little bug snuck through our testing? You just flip the load balancer back to Blue. The rollback is seconds, not hours.

Defining Blue-Green Infrastructure

To get this working within AWS using Terraform you need to create two target groups and let ECS manage the traffic moving.

# The target group for our current live traffic
resource "aws_lb_target_group" "primary_tg" {
  name        = "app-primary-tg"
  port        = 80
  protocol    = "HTTP"
  vpc_id      = aws_vpc.main_network.id
  target_type = "ip"

  health_check {
    path                = "/api/health"
    interval            = 15
    healthy_threshold   = 2
  }
}

# The target group for our incoming new releases
resource "aws_lb_target_group" "secondary_tg" {
  name        = "app-secondary-tg"
  port        = 80
  protocol    = "HTTP"
  vpc_id      = aws_vpc.main_network.id
  target_type = "ip"

  health_check {
    path                = "/api/health"
    interval            = 15
    healthy_threshold   = 2
  }
}

You set up your ECS service to use the CODE_DEPLOY controller (or native ECS deployment tools) to handle the transition between the two target groups.

resource "aws_ecs_service" "web_api" {
  name            = "core-web-api"
  cluster         = aws_ecs_cluster.production.id
  task_definition = aws_ecs_task_definition.api_def.arn
  desired_count   = 4

  deployment_controller {
    type = "ECS"
  }

  # Instructing ECS to keep the old tasks around for a safety buffer
  blue_green_deployment_config {
    deployment_ready_wait_time_in_minutes = 10
    terminate_blue_tasks_on_deployment_success {
      enabled               = true
      termination_wait_time = 30
    }
  }
}

The Taste Test: Canary Releases

While Blue-Green provides a good safety net, it still requires all your users to move to the new version at the same time.

And here’s where it gets interesting...what if you only exposed a very small fraction of your consumers to the new code?

Think of it as making a big pot of soup for a banquet. You taste one spoonful, then serve it to 500 visitors. You put sugar instead of salt in the soup; by accident, you only spoiled one spoonful, not the whole dinner party.

Canary deployments slowly transfer traffic to the new version in small increments.

The Phased Rollout Process

Instead of a hard swap, ECS’s canary deployment uses the weighted routing features of an AWS Application Load Balancer.

Phase 1: The Initial Dip. You push the new version out, but only send 5% of your live traffic to it. The other 95% continue with the stable version.
Phase 2: Metric Monitoring. You keep a careful eye on your dashboards. Is that 5% seeing error rate spikes? Seeing database slowness growing up?
Phase 3: The Ramp-Up. If the system seems healthy after a period of time, you bump the weight up to 20%, then 50% and finally 100%.

If the new version faults out at any stage, you immediately set the ALB weights for the new version back to 0%. The bug’s blast radius is tightly controlled to a small subset of users.

Configuring Weighted Traffic

To do this, you create a listener rule on your AWS load balancer to route traffic based on the supplied weights.

resource "aws_lb_listener_rule" "canary_traffic_split" {
  listener_arn = aws_lb_listener.https_listener.arn
  priority     = 50

  action {
    type = "forward"
    forward {
      target_group {
        arn    = aws_lb_target_group.stable_version.arn
        weight = 90
      }
      target_group {
        arn    = aws_lb_target_group.new_release.arn
        weight = 10
      }
      # Critical: Ensure users don't bounce between versions
      stickiness {
        enabled  = true
        duration = 3600 
      }
    }
  }

  condition {
    path_pattern {
      values = ["/api/*"]
    }
  }
}

See the stickiness configuration in the codeblock. This ensures that a user who lands on the updated version of your app stays on that version for the length of their session. Randomly bouncing a user between two separate codebases every click will be a poor user experience.

Choosing Your Deployment Weapon

Fair enough... neither is always "better" than the other. They encounter quite different operating challenges.

Opt for Blue-Green when:

You are installing internal tools or APIs where a harsh cutover is tolerable.
You want to be able to conduct intensive integration tests on the exact production infrastructure before any real traffic actually hits the servers.
You want your deployment pipeline to be simple and speedy.

Opt for Canary when:

You are deploying really critical functionality (like payment gateway) where even 1 minute outage is fatal.
You need to validate business metrics (e.g. conversion rates or user engagement) and technical data before a full launch.
If the traffic volumes are high, then you will have enough data even with a small 2% slice of traffic to identify anomalies.

Rules for Safe Deployments

Whichever approach you pick, dynamically rerouting traffic adds layers of complication. You can't just lay down in these infrastructure models and expect miracles if you disregard the rest of the ecosystem.

1. The Database Schema Trap

Simply said, your database is not rollback-able, but your code is.

If you change a database column name in your new deployment and direct traffic to the new version, the previous version will immediately crash because it is expecting the old column name. If you ever have to roll back, your application is now broken forever.

If you want zero-downtime deployments then database modifications must be fully backward-compatible. You have to follow a pattern of “expand and contract”:

Deploy 1: Add the new database column (both code versions still function).
Deploy 2: Update the application code to read/write to the new column.
Deploy 3: Remove the old column days later, long after the rollback window has closed.

2. You Must Have Excellent Metrics

Driving blindfolded is like moving traffic without visibility.

How do you know a Canary release is working when you route 10% of your traffic to it? You can’t depend on users to file support tickets. You want automatic dashboards on HTTP 500s, response times, and CPU utilization.

3. Automate the Rollback

Humans panic. Systems do not.

Don’t have a stressed engineer manually modify load balancer weights when anything goes sideways. Use automatic lifecycle hooks. You can use AWS to trigger Lambda functions on ECS deploys. Your Lambda can automatically abort the deployment and roll back traffic if it finds that the new target group is failing health checks.

Key Takeaways

Blue-Green deployments enable an easy escape route by keeping two similar settings that offer a suitable testing ground before going live.
Canary releases minimize the damage of uncovered bugs by first exposing new code to a small subset of users and growing up gradually.
It must be able to work with databases backward. If your database updates break your previous code, you don't have your safety net of rollback anymore.
Observability is a must. If you don't have analytics to see how your application is functioning in real-time, you can't securely move traffic.

Conclusion

Software deployments should be dull, at the end of the day.

By using Blue-Green cutovers for a speedy escape route, or Canary rollouts to reduce the blast radius of issues, you safeguard your users from your mistakes. When engineers are no longer afraid to break the system, they code with more confidence. They merge pull more quickly. They provide little, digestible updates rather than terrifyingly large monoliths.

Long story short...investing time in your ECS deployment architecture is more than simply focusing on server health. It’s about maintaining the mental health of your engineering staff, keeping your weekends intact, and delivering uninterrupted value to your users.

About the Author

As an AWS Community Builder, I enjoy sharing the things I've learned through my own experiences and events, and I like to help others on their path. If you found this helpful or have any questions, don't hesitate to get in touch! 🚀

🔗 Connect with me on LinkedIn

Also Published On

AWS Builder Center

Hashnode

Taming AWS with Claude: Why Your AI Needs a Memory

N Chandra Prakash Reddy — Mon, 17 Aug 2026 11:27:49 +0000

Let’s be real. We all know the first magic of AI coding helper. You ask a query, and voila! A nicely formatted script pops up.

But if you work in current cloud tech, that magic wears off fast. You start a new browser tab and ask for help with your cloud infrastructure and the AI quickly proposes a generic configuration. It tells you to open your security groups to the whole internet, or it uses the wrong cloud region, or it entirely ignores your team’s rigorous naming rules.

That sounds familiar? This is the frustrating reality of working with ordinary AI tools. They are suffering from severe amnesia. As soon as you close the session, they forget all about your architecture.

Now here's where things gets interesting. A new breed of AI tools such as Claude Code are profoundly changing this dynamic. They don’t sit in your browser as a passive chatbot; they sit in your terminal. They can read your local files, run commands and most of all, they truly remember how your individual project is wired.

Why is this context-aware strategy totally rewriting the playbook for developers, ops teams and security engineers? Let’s dig deeper.

The Core Concept: Giving Your AI a Permanent Memory

Context is the biggest challenge in cloud engineering. Your organization doesn’t just use “the cloud” - you use a very unique and highly customized version of it.

You may need to have your entire infrastructure as terraform code. You can deny wildcard rights explicitly in your access policies. Maybe a compulsory labeling system to track billing .

If you have to explain this every single day to an AI, you are losing precious time. To be fair, regular AI models aren’t attempting to be difficult, they simply don’t have access to your environment.

Terminal-native agents do this with a local memory file - commonly a plain markdown file lying directly at the root of your codebase. Consider this PDF as an onboarding guidebook for a new hire. The AI reads this guidebook every time it wakes up.

Here’s an example of what an original, very particular context file may look like for a fictional payments service:

# project-context: payment-gateway-api

## Cloud Environment Rules
- Primary Cloud: AWS
- Default Region: eu-central-1 (Frankfurt)
- compute: We only use AWS Lambda (Node.js 20.x runtime). Do not suggest EC2 or containers.
- Databases: DynamoDB for transactions, Redis for caching.

## Security & Compliance
- NO hardcoded secrets. Ever. Fetch everything dynamically from AWS Parameter Store at runtime.
- IAM Policies: Strictly least-privilege. Never use "*" for resources or actions.
- Network: All outbound traffic must route through our NAT Gateway.

## Development Standards
- Infrastructure as Code: We strictly use AWS CDK (TypeScript). No Terraform.
- Testing: Jest for unit tests. Minimum 90% coverage required for PRs.
- CI/CD commands: 
  - Build: `npm run build`
  - Test: `npm run test`
  - Synth: `npx cdk synth`

This is the kind of file you have in your repo to set the ground rules. Any developer on your team can invoke the AI and the result will automatically stick to your tight corporate requirements. No more entering your cloud region and language choices manually.

How Developers Actually Use Context-Aware Agents

But what does this really look like in practice? The basic logic of an application is normally the fun portion to write for a feature developer. The boilerplate is the tiring part. Getting the event triggers wired up, setting up the cloud permissions, writing the deployment scripts.

For instance, say you want to build a new background worker to handle user uploads. With a normal AI, you’d ask for the application code, then ask for the infrastructure code, then spend an hour making sure they really spoke to each other.

If you already have a terminal-based agent that reads your context file, your prompt can be quite short:

claude -p "Generate a new background worker called 'image-optimizer'. It needs to trigger whenever a new file lands in our raw-uploads S3 bucket. Compress the image, save it to the processed-uploads bucket, and log the event to our DynamoDB tracking table. Include the full AWS CDK stack and the Jest tests."

Simply put, you are the designer, The AI builds the TypeScript code, creates the buckets, writes the least-privilege access controls, and mocks the cloud services for your local testing. The code it generates knows your rules, so it's actually usable right out of the start.

The DevOps Reality: Automating the Troubleshooting

For operations and platform teams, the worst part of the job isn't constructing things, it's figuring out why things randomly stopped working.

When a continuous integration (CI) pipeline fails, or a cloud deployment gets stuck in a rollback cycle, a human engineer often has to drop everything. They have to navigate through hundreds of lines of unusual cloud logs to figure out that one environment variable was missing.

The issue is, terminal-native AI is able to execute in your automated workflows (like GitHub Actions) without human interaction. You are able to set up a routine to automatically trigger when a deployment fails.

Instead of paging an engineer at 2:00 AM , the pipeline can tell the AI to go check it out . Here is an example of an original structure for an automated debugging action:

name: Auto-Triage Failed Deployments
on:
  workflow_run:
    workflows: ["Production Deployment"]
    types: [completed]

jobs:
  investigate-failure:
    if: ${{ github.event.workflow_run.conclusion == 'failure' }}
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v4

      - name: Authenticate Cloud Provider
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.DEBUG_ROLE_ARN }}
          aws-region: eu-central-1

      - name: Trigger AI Investigation
        uses: some-ai-provider/cli-action@v2
        with:
          api_key: ${{ secrets.AI_API_KEY }}
          instructions: |
            Our recent Serverless deployment just failed. 
            1. Fetch the latest CloudFormation stack events for 'payment-gateway-prod'.
            2. Pull the last 20 minutes of CloudWatch logs for the deployment function.
            3. Identify the exact resource that caused the rollback (e.g., IAM permission boundary issue, timeout, missing parameter).
            4. Create a pull request with the necessary code fix and a plain-English explanation of what went wrong.

The AI performs the read-only commands, interprets the encrypted error messages, and provides a fix. The human engineer has still the last word, but the drudgery of investigation has been done.

Shifting Security Left

Security teams are commonly referred to as the “Department of No,” since they usually discover misconfigurations at the very end of the development process.

Terminal AI changes this - it is a localized security watchdog. The AI understands your application logic and your cloud infrastructure, so it can detect risky patterns even before code is submitted.

You can enforce local configuration hooks to intercept commands. The system stops the AI (or a developer) from doing anything hazardous.

Here’s an example custom configuration snippet to prevent catastrophic removals of infrastructure:

{
  "safety_hooks": {
    "before_execution": [
      {
        "trigger": "terminal_command",
        "pattern": "(terraform destroy|aws s3 rm --recursive)",
        "action": {
          "type": "block",
          "error_message": "CRITICAL: Destructive cloud commands are disabled in this project. You must perform this action manually via the console with secondary approval."
        }
      }
    ]
  }
}

This assures that the artificial intelligence cannot unintentionally delete a database or storage bucket, no matter what prompt the user gives it.

You may script the AI to do huge automatic audits as well. You could construct a simple bash loop that the artificial intelligence can use to go thru your codebase, looking at every single identity policy, and flag any permission that allows access to all resources. This reduces a multi-week manual audit to a five-minute automated scan.

Standard Chatbots vs. Terminal Agents

If you are still undecided, here is a one-sentence method to think about the difference between the two paradigms:

Feature	Browser-Based AI	Terminal-Native Agent
Awareness	Only knows what you manually type into the chat box.	Can read your file tree, configuration files, and scripts.
Actionability	Gives you code snippets to copy and paste.	Executes shell commands, formats files, and runs your test suite.
Consistency	Requires you to re-explain your architecture every day.	Automatically inherits your team's persistent markdown rules.
Troubleshooting	Requires you to manually paste error logs into the chat.	Can actively query cloud APIs to find the error logs itself.

Putting Up Guardrails

It’s a powerful technology, but we need to be realistic about safety. You are giving an automated system access to your terminal and possibly your cloud environment.

If you do decide to proceed with this method, there are a few regulations you have to adhere to, no matter what. First, long-lived, static access keys are never allowed. Always utilize temporary, auto-rotating credentials with strong least privilege roles.

Second, the AI should not have the power to merge its own code or deploy directly to production. AI suggests a solution. Human approves the solution.

Key Takeaways

If you're ready to level up your cloud workflows, these are the basic ideas to keep in mind:

Memory beats intelligence: A little less powerful AI that knows exactly how your AWS VPC is laid out is far more beneficial than a super-intelligent AI that thinks you are building a generic educational app.
Boilerplate is for bots: No more hand-writing IAM policies and basic cloud scaffolding. Let the context aware agent worry about the plumbing, while you work on the business logic.
Automate the triage: CI/CD pipelines should not merely tell you that a deployment has failed. Your pipelines with AI agents may search the cloud logs, determine the root cause, and write the solution.
Security must be proactive: Pre-execution hooks and automatic audits let you identify glaring AWS misconfigurations right in the terminal, long before they become a big issue for the security team.

Conclusion

The days of copy-pasting the same general code in a tab from the browser are over.

You put AI right into the terminal and give it the persistent memory of your infrastructure, removing the blank slate tax. It helps developers move faster, helps DevOps teams quickly resolve complicated AWS issues, and provides security teams with a proactive mechanism to detect vulnerabilities early.

After all, creating and scaling apps in the cloud is complex enough. A digital assistant that truly remembers the way your personal environment operates is no longer a luxury. It’s the fastest, smartest way to design robust software.

About the Author

🔗 Connect with me on LinkedIn

Also Published On

AWS Builder Center

Hashnode

Control AWS Traffic: Smart Serverless Throttling

N Chandra Prakash Reddy — Mon, 10 Aug 2026 09:31:12 +0000

When I began building cloud-native apps on AWS, I was really focused on writing business logic. I liked how easy it was to connect API Gateway to AWS Lambda and get a working microservice up and running in just a few minutes.

Let’s face it, when we’re focused on launching new features, things like request caps and execution limits usually get pushed to the bottom of our to-do list.

Does this sound familiar? Many engineering teams only think about traffic control after their backend databases crash or they get a huge, unexpected AWS bill. If you’re wondering how to avoid these problems before they become late-night emergencies, I’ve put together the framework I use for traffic throttling with Terraform, based on what I’ve learned from improving my own deployment workflows.

The Hidden Danger of Uncontrolled Cloud Traffic

While serverless service like AWS Lambda can scale on their own, your databases, payment systems, and third-party APIs often cannot keep up with unlimited traffic.

Why Every Developer Needs a Traffic Cop

Think about a popular amusement park ride with only 20 seats. If 5,000 excited visitors all try to get on at once, it turns into chaos. People get pushed, the entrance gets blocked, and no one gets to enjoy the ride.

Request throttling works like a velvet rope at the entrance. It allows a steady, manageable group of visitors in, while keeping the rest waiting safely in line. Setting clear limits protects your backend services, keeps performance steady, and helps guard against accidental loops or attacks that try to overload your system.

Building a Production Defense Line with Terraform

Here’s where things get interesting. Instead of manually changing settings in the AWS Management Console, which is often error-prone and hard to track, we can manage all our traffic rules as code using Terraform.

Customizing Limits per Deployment Tier

Each environment has its own operational needs. For example, your staging environment should shut down early to avoid runaway integration tests, while your production tier needs enough capacity to handle real user surges.

We can do this easily by setting up environment-aware variables in Terraform:

variable "app_stage" {
  type        = string
  description = "Deployment target name (e.g., dev, prod)"
}

variable "gateway_rate_configs" {
  type = map(object({
    peak_burst   = number
    steady_rate  = number
  }))
  default = {
    dev = {
      peak_burst  = 500
      steady_rate = 250
    }
    prod = {
      peak_burst  = 3000
      steady_rate = 1500
    }
  }
}

Preventing Lambda Resource Hijacking

By default, each AWS account has a regional pool of 1,000 concurrent Lambda executions. If one unoptimized background worker uses too many resources, it can use up the entire quota and cause your important public APIs to stop working.

To stop a single function from using all your resources, you can set aside dedicated concurrency slots:

resource "aws_lambda_function" "order_processor" {
  function_name = "order_processor_${var.app_stage}"
  # ... standard lambda configurations ...

  reserved_concurrent_executions = lookup(var.concurrency_caps, var.app_stage, 75)
}

If you set a limit on concurrency, you make sure this function does not use up all your account capacity or put too much pressure on your database connections.

Gating Your Front Door with Usage Caps

Amazon API Gateway acts as the entry point for your microservices. When you use stage settings together with usage plans, you can control both short-term bursts and total monthly usage for people using your API.

resource "aws_api_gateway_usage_plan" "tier_policy" {
  name = "client-access-plan-${var.app_stage}"

  api_stages {
    api_id = aws_api_gateway_rest_api.core_api.id
    stage  = aws_api_gateway_stage.live_stage.stage_name
  }

  quota_settings {
    limit  = 25000
    period = "MONTH"
  }

  throttle_settings {
    burst_limit = lookup(var.gateway_rate_configs[var.app_stage], "peak_burst", 500)
    rate_limit  = lookup(var.gateway_rate_configs[var.app_stage], "steady_rate", 250)
  }
}

Keeping Your Infrastructure Safe and On Budget

Enforcing request limits is just one piece of the puzzle. You also need clear insight into your spending and traffic health so you know exactly what's going on.

Catching Oversights with Automated Budget Guards

Traffic spikes shouldn't catch you off guard with unexpected bills. By setting up an automated AWS Budget with Terraform, your team will get alerts well before costs get out of control.

resource "aws_budgets_budget" "account_cost_cap" {
  name         = "monthly-spend-guard-${var.app_stage}"
  budget_type  = "COST"
  time_unit    = "MONTHLY"
  limit_amount = "750"
  limit_unit   = "USD"

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "FORECASTED"
    subscriber_email_addresses = ["ops-team@mycompany.com"]
  }
}

Setting Up Real-Time Warning Signals

Besides financial alerts, it’s important to track operational metrics. For example, if your rate limits are too strict and real users get blocked, a CloudWatch alarm can alert you right away.

resource "aws_cloudwatch_metric_alarm" "lambda_breach_warning" {
  alarm_name          = "lambda-throttled-${aws_lambda_function.order_processor.function_name}"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 2
  metric_name         = "Throttles"
  namespace           = "AWS/Lambda"
  period              = 300
  statistic           = "Sum"
  threshold           = 5
  alarm_actions       = [aws_sns_topic.operations_alert.arn]

  dimensions = {
    FunctionName = aws_lambda_function.order_processor.function_name
  }
}

Dynamic Scaling Strategies That Work in the Real World

Static infrastructure rules can seem too rigid when compared to the changing needs of real applications. Most apps go through busy periods in the morning and quieter times at night.

Shifting Traffic Capacity on a Schedule

There’s no need to pay for extra capacity when it isn’t needed at night. Instead, you can set up AWS EventBridge to run a simple Python script that changes API Gateway limits as needed during the day.

Below is a basic Python function you can schedule to run automatically:

import boto3
import os

def sync_rate_limits(event, context):
    apigw = boto3.client('apigateway')

    target_api_id = os.environ['REST_API_ID']
    target_stage = os.environ['STAGE_NAME']
    new_burst = int(os.environ['TARGET_BURST'])
    new_rate = int(os.environ['TARGET_RATE'])

    apigw.update_stage(
        restApiId=target_api_id,
        stageName=target_stage,
        patchOperations=[
            {'op': 'replace', 'path': '/*/*/throttling/burstLimit', 'value': str(new_burst)},
            {'op': 'replace', 'path': '/*/*/throttling/rateLimit', 'value': str(new_rate)}
        ]
    )

    return {'statusCode': 200, 'body': 'Updated API limits successfully.'}

Set this script to run at 8:00 AM to open more traffic lanes for the morning rush. Then, use a matching rule at 8:00 PM to reduce the lanes again when traffic is lighter.

Filtering Out Noise with Edge Security

In short, throttling helps protect your internal application logic, but it does not prevent malicious bots from using up your network bandwidth.

Adding an AWS Web Application Firewall (WAF) to your API Gateway gives you strong protection at the edge. When you use AWS Managed Bot Control rules with Terraform, you can block unauthorized scrapers and spam traffic before they reach your application code.

Key Takeaways

Before you start working in your own AWS environment, here’s a quick summary of the key concepts we discussed.

Treat limits as code: Define your API Gateway and Lambda throttling rules with Terraform. This helps keep things consistent and reduces mistakes.
Protect the backend: Set a limit on your Lambda concurrency. This way, one problematic function can’t take over your account or overload your database.
Stay ahead of the bill: Set up automated AWS Budgets and CloudWatch alarms. These tools help you spot traffic spikes early and avoid unexpected costs.
Scale intelligently: Use EventBridge to change capacity depending on the time of day. This keeps performance up when it’s busy and saves money during quieter times.
Block the bots: Add AWS WAF to block harmful traffic before it reaches your compute resources.

Conclusion

Managing serverless performance is something you need to keep up with, not just set up once. Setting request limits does not hold back your application's growth. Instead, it helps you build a system that stays strong and reliable, no matter how much traffic comes its way.

When you add these safety rules to your Terraform setup, you protect your backend, save on cloud costs, and make sure every user has a smooth experience. Now you have a clear plan to keep your cloud workloads safe, scalable, and fully under your control.

About the Author

🔗 Connect with me on LinkedIn

Also Published On

AWS Builder Center

Hashnode

Locking Down Your Cloud: A Beginner's Guide to AWS KMS

N Chandra Prakash Reddy — Sun, 09 Aug 2026 10:40:30 +0000

A couple of months ago I worked on a side project, a local food delivery service. We were moving fast, creating features, integrating payment gateways. One evening I found myself doing something terrifying: I was going to put our main database encryption password directly into our configuration file.

Let’s be honest, we have all been tempted to choose the easy road. But here’s the problem: if I had uploaded that file to GitHub, a scraping bot would have been able to find that password in less than five seconds. From there, anyone could have accessed our database, decrypted it and walked away with hundreds of customer addresses and phone numbers.

That close call got me thinking about how I handle application secrets. That got me looking into AWS Key Management Service (KMS).

In this article, I’ll walk you through how to properly protect an application with AWS KMS. We’ll skip the textbook definitions and walk through a real world scenario to show you exactly how to keep your users data locked down.

The Danger of the "Hidden" House Key

But before we talk about the cloud, let’s talk about how traditional encryption normally goes wrong.

Imagine you purchase an impenetrable safe to house your life wealth. But rather than memorising the combination you write it down on a sticky note and put it to the side of the safe. That’s exactly what occurs when developers implement their own encryption logic and hardcode their cryptographic keys into their application source code.

If an attacker gets access to your code, through a leaked github token, a frustrated worker, or a hole in your server, they get the key immediately. They don't have to break the encryption, they just step right in the front door.

The solution is a system which does not allow the key to be in the same place as the code or the data.

The Digital Bank Vault: Enter AWS KMS

AWS KMS is like a high security bank vault.

If you wish to deposit your assets with the bank, they don't offer you the master key to the vault. Instead, you walk up to the teller, hand over your items and show your identity. The teller takes your stuff into the vault, puts it in a box and gives you a receipt. When you want your stuff back you show the receipt and your ID and the teller brings your items out.

AWS KMS works in precisely the same way. It is a fully managed service and is your digital bank teller. It creates and securely stores top-level encryption keys (called KMS Keys) in AWS hardware that is designed for this purpose. The raw encryption key is never seen or touched. Instead, your application asks KMS to encrypt or decrypt data for it.

Why This Approach Wins

Zero Key Leakage: Your application code never actually touches the raw key so you can’t unintentionally push it up to a public repo.
Seamless Ecosystem: It has immediate integration with services such as Amazon S3, RDS ( databases ) and EBS ( hard drives ) . So you can typically encrypt your cloud storage with one click .
Always Online: KMS is designed for high availability. No need to worry about your application hitting a “vault” closure during a late night traffic surge.

The Bouncers at the Door: Controlling Access

So you might be asking yourself, if all our keys live in AWS how can we block some bad inside app from telling KMS to unlock everything?

AWS solves this by demanding two different levels of permissions. Imagine a really exclusive VIP club with two distinct gatekeeper at the main door. You have to go through both to get in.

1. The IAM Policy (The Guest List)

The first gatekeeper is the IAM (Identity and Access Management) policy. This is linked to your application or your developer account. It controls what the user can do with AWS in general. Your back-end server must have an IAM policy that allows it to talk to KMS, otherwise the first bouncer will turn it away right away.

2. The Key Policy (The VIP Pass)

The Key Policy is the second bouncer. This policy is tied directly to the encryption key . Even if a developer has global admin access in IAM, if the particular Key Policy reads, “Only the billing microservice can use this key,” the developer gets rejected.

To be fair, it is a little laborious to juggle two sets of regulations when you are starting off. But the final result is that if one of your servers ever gets hacked, this “two-bouncer” technique contains the blast radius.

In Action: Securing a Customer's Checkout Data

Now lets put this in perspective with a real world example. Remember the food delivery app I was telling you about?

We want to encrypt the home delivery address when a consumer enters it, before we save it in our database. We will leverage the AWS SDK for Python (Boto3) to request that KMS protect the data.

Step 1: Encrypting the Address

So when the user clicks save , our backend gets the raw address and transmits it directly to the KMS service.

import boto3

# Connect to the AWS KMS service
kms_client = boto3.client('kms')

# The sensitive data from our user
user_address = b'123 Main Street, Apartment 4B'

# Ask KMS to lock it up using our specific Key ID
response = kms_client.encrypt(
    KeyId='arn:aws:kms:us-east-1:123456789012:key/your-unique-key-id',
    Plaintext=user_address
)

# KMS hands us back a scrambled, unreadable blob
scrambled_data = response['CiphertextBlob']
print(scrambled_data)

Now we can securely take the scrambled_data blob and save it in our database. So even if a hacker dumps the whole database tables they will see a huge number of random worthless characters.

Step 2: Decrypting the Address for the Driver

When the delivery driver accepts the order our software has to read the address. We take the scrambled blob from the database and return it to KMS.

Note that we don't even have to tell KMS which key to use here, it just knows automatically based on hidden metadata inside the blob!

# Ask KMS to unlock the data
response = kms_client.decrypt(
    CiphertextBlob=scrambled_data
)

# Extract the original, readable address
readable_address = response['Plaintext']
print(readable_address.decode()) 
# Outputs: 123 Main Street, Apartment 4B

KMS returns the plaintext if the server running this code has the correct IAM and Key rules.

Changing the Locks: Key Rotation and Aliases

Security is not a “set it and forget it” thing. And just like you should change the locks on a physical structure every once in a while, you should rotate your encryption keys.

When you’re doing your own cryptography, rotating a key is a nightmare. You have to stop your program , decrypt your entire database with the old key , re-encrypt it with the new key , and pray nothing crashes .

In AWS KMS there is a button that says “Enable automatic key rotation.” You hit it. Every year AWS will generate a completely new key, secretly, and we will utilise that for all future encryption. Best part? It remembers the old keys forever so it can still decrypt your old database records without you changing a line of code.

A Quick Tip: Use Key Aliases

You spotted a big arn:aws:kms… string in the Python code above. Hardcoding those large strings might get ugly. KMS allows you to construct friendly names called Aliases (like alias/delivery-app-key).

In an emergency, if you need to point your app to a totally different master key, you just need to change what the alias points to in the AWS interface. Your code stays neat and clean, and the changeover is instantaneous.

The Security Cameras: Auditing with CloudTrail

You might be wondering, “How do I know if someone is trying to misuse my keys?

Now it’s becoming pretty fascinating. AWS KMS is tightly connected with another service called AWS CloudTrail. Imagine CloudTrail as a series of invisible security cameras that keep an eye on your digital bank vault.

When your application (or a developer) requests KMS to encrypt or decrypt information, CloudTrail tracks it. If you suspect a compromise, you can open CloudTrail and receive a comprehensive receipt: 02:04 AM - User X attempted to decrypt data using Key Y and IP Address Z. It is a total lifesaver for passing compliance checks or investigating suspicious behaviour.

Key Takeaways

If you’re designing a modern application, managing your own encryption keys manually is an unnecessary risk. What you should remember is:

Never hardcode secrets: The real cryptographic key should never be in the source code of your application.
Embrace two-layer security: Use both IAM policies and KMS Key policies together to tightly control which applications can access your keys.
Automate rotation: Enable automatic key rotation in AWS KMS. It protects future data smoothly. And at the same time it is backward compatible with old data.
Use aliases: Use Key Aliases instead of large ARN strings to make your code clearer and easier to manage.
Audit everything: Use AWS CloudTrail to track exactly who is using your keys and when so you always have a full security history.

Conclusion

Cryptography can be really complicated stuff and, in the end, trying to design your own security system from scratch is a huge organisational risk.

AWS Key Management Service simplifies the most challenging portions of cryptography, including secure physical storage, hardware maintenance, and transparent key rotation, into secure API requests. KMS is the ideal solution for the job, whether you are a solo developer trying to protect your first few user passwords, or a big technical team locking down a corporate health platform.

By using centralised keys, stringent access control, and automated rotation, you greatly minimise your risk. Stop hiding your digital house keys under the doormat. Let AWS hold the vault.

About the Author

As an AWS Community Builder, I enjoy sharing the things I've learned through my own experiences and events, and I like to help others on their path. If you found this helpful or have any questions, don't hesitate to get in touch! 🚀

🔗 Connect with me on LinkedIn

Also Published On

AWS Builder Center

Hashnode

LamRAG: 800GB, AI, And Lessons From A Project We Couldn't Complete

N Chandra Prakash Reddy — Sun, 19 Jul 2026 12:21:13 +0000

Let’s face it, tech conferences are usually a bunch of polished success stories, where everything works out. But one of the presentations at the AWS Community Day Kochi on December 20, 2025 took a refreshing turn.

The event had many fantastic sessions but the one which really stole the show for me was Tech Session provided by Sandeep Kumar Prakash. The title alone was a hook: “LamRAG: 800GB, AI, And Lessons From A Project We Couldn’t Complete.”

It was a masterclass in what happens when big AI dreams run into the hard realities of big data pipelines. If you’ve ever tried to design a generative AI application and felt like you were running into a brick wall, this story is for you.

The Hackathon Dream: Managing a Valorant Roster

The project started off as a hackathon focused on Valorant. In case you don't know, Valorant is a very competitive online multiplayer First Person Shooter (FPS). It’s a 5v5 team arrangement where players can select from 28 different agents and fight on 12 different maps. The stakes are high, with 13 rounds in each game and one life per round for players.

The team's mission sounded simple enough on paper. They wanted to play the role of a team manager. The goal was to construct a chatbot that could develop plans and manage league or region based team identification.

To do this, they needed data. Lots of it.

Data Infrastructure: When Big Data Fights Back

To begin, the team collected three years of data from Valorant games from three separate leagues, for a total of 7,357 individual games. So, all these files were stored by the Amazon S3. They were organised in directories like “vct-challengers”, “vct-international” and “game-changers”.

At start, the total size of the S3 bucket was 101.2 GB, compressed data. That sounds straightforward, right?

The problem with data is that it can be misleading. When they decompress the files, the 101 GB of compressed data increased to an awesome 800 GB of uncompressed data.

They instantly ran into a big barrier. The raw data came in the form of very thick json files. Speaker offered showed a single JSON file with 8.3 million lines of code filled with complex server info and metadata. Trying to open these enormous files locally absolutely crashed their development environments.

The team had to pivot their work flow totally to remedy this. They eliminated local processing and went with amazon lightsail. Amazon Lightsail and VS Code’s Remote-SSH features allowed them to finally get back to a functional and pleasant developer experience without their PCs exploding into flames.

From Raw Data to Strategy: Finding the Signal in the Noise

The team rapidly learned a critical lesson: “Volume!= Value”. The fact that you have 800GB of data does not indicate that it is all valuable to an AI model.

Imagine you are searching for a certain recipe in a huge library. If you just pick up every single book and throw it on your desk you are going to feel overwhelmed. You need an index.

To tackle this, they implemented a data sampling method with Amazon EC2. They started moving data from their ‘raw’ folders to ‘sampling’ folders by year: 2022, 2023, 2024.

Rather than sending the complete game logs into an AI, they started to separate and characterise the data. They isolated some states of the game, including "GAME_STARTED", "ROUND_STARTING", "IN_ROUND" and "GAME_ENDED". After some careful parsing, this chaotic 800GB mess was turned into highly organised, focussed datasets with unambiguous names like map_agent_rounds_stats.json and top_3_combinations_per_map.json.

The data was finally clean, it was structured, it was ready. But as speaker challenged the audience, “Data is ready but how do we use it?

The Knowledge Base Attempt: Why Default RAG Failed

The logical next step for most developers is Retrieval-Augmented Generation (RAG).

So the team turned to Amazon Bedrock to develop a Knowledge Base by connecting straight to their S3 data-source called valorant-player-agent-map-data. They had to decide how to “chunk” or break up the text for the AI. They compare a "Default Chunking" method with a "No Chunking" method, where the data is chunked into vector records of 300 tokens while in the other a complete file is supplied as a single vector record.

They booted up the Claude 3.5 Sonnet base model to try out their bright new Knowledge Base. The prompt was simple: 'make the best squad depending on combat score'.

Now this is were it gets interesting. The AI went crazy.

The reply said there was a player named Meteor with a fighting score of 112,861. Another player, Lakia, was reported to have scored 68,939. Any Valorant player knows these figures are mathematically insane.

The Culprits: Context Hallucination and Numeric Drift

So why did a super powerful AI fail fundamental statistics? Speaker set out two big ideas that all AI developers need to know:

Context Hallucination: Traditional embedding models drop context. The AI didn't understand what a "combat score" actually meant in terms of "damage given". It basically searched for text strings that lived nearby in the database.
Numeric Drift: Vectorising numbers makes them lose their arithmetic value. The numbers are stored only according to their logical context and not their numerical weight.

It was just treating " 112,861 " as if it were a word in a sentence , not a number that should be sorted or calculated.

The Pivot to Agents and Function Calling

Since classic RAG couldn’t handle maths, the team asked a new challenge, “How do we let AI use all the metrics?”

The answer? Function Calling.

Instead of making the LLM read static text, they constructed an Amazon Bedrock Agent called valorant-l4-fn-calling. Again, they used Claude 3.5 Sonnet, but this time they gave it clear instructions that defined the player’s identification (handle, first name, last name) and core metrics (Assists, CombatScore, Games_played). They also formed specific “Action groups” for assignments.

The architecture was fully changed to a much more dynamic system:

Amazon Bedrock acts as the brain.
The LLM invokes an AWS Lambda function with a produced DSL (Domain Specific Language) Query.
AWS Lambda runs the same DSL query against an Amazon OpenSearch database.
OpenSearch returns the hard, calculated data back to Lambda.
Lambda cleans up the data and sends it back to the LLM to format into a human readable answer.

Suppose you order some meals on Swiggy. You’re not asking the app to prepare the food (conventional RAG tries to do the maths). You utilise the app to send a structured request (Function Call) to a restaurant (OpenSearch) and the delivery driver (Lambda) returns the identical result to you.

Cognitive Overload: The Final Roadblock

The architecture was magnificent, but the team ran across one final, impenetrable obstacle.

They were asking the LLM to accomplish far too much at once. The task was to locate the best characters for N maps, then find the best players for these characters, take 5 players and match them to characters on maps, and finally build a strategy.

Speaker called this "Cognitive Overload".

The technical problems came thick and fast:

The Action Group answers could not be more than 25 KB, they had a strict limit.
The system was calling functions many times in the same loop.
Sometimes, the LLM created invalid OpenSearch searches.
Eventually this resulted in a “Frozen” LLM that just hung up and stopped responding altogether.

As the presentation very well concluded, “Everything worked until it worked together.”

Key Takeaways

If you are planning to construct a generative AI application, here are the lessons to learn the most:

Volume does not equal value: 800GB of raw JSON at a problem won’t solve it. You have to clean, sample and format your data before the artificial intelligence can use it efficiently.
Standard RAG can't do math: The point of embedding models is logical similarity, not mathematical accuracy. If you need to sort numbers or discover a “highest score”, by default vector search will likely hallucinate due to numeric drift.
Function calling is your bridge: Instead of having the LLM read a database, employ tools like AWS Lambda and OpenSearch to allow the LLM to query the database.
Beware of cognitive overload: Asking an AI to do complex filtering, matching and generating all in one single command will ruin your system. Divide complicated work into smaller, manageable chunks.

Conclusion

This was a tremendous eye-opener of a session toward the conclusion of the day. We generally prefer to conceive of AI as a sort of magic wand that we can wave over unstructured data and obtain accurate outcomes.

The biggest lesson Sandeep presented was “The Slap On The Face: Don’t teach an LLM what it already knows. If you’re creating AI apps, don’t make a language model do significant statistical calculations or analyse raw databases. Allow the LLM to do what it is best at reasoning and language and leverage function calls to offload the heavy job to your traditional databases.

It was an extraordinary experience to observe the rough, unfinished, and incredibly informative side of building on AWS." Sometimes the tasks we don’t get to finish teach us much more than the ones that go perfectly.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: LamRAG: 800GB, AI, And Lessons From A Project We Couldn't Complete

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

AWS Bedrock VPC Flow Log Analyzer

N Chandra Prakash Reddy — Sun, 19 Jul 2026 08:20:31 +0000

I got a chance to attend AWS Community Day Kochi on 20 December 2025. What an amazing event! Brilliant minds, deep-dive technical courses and a strong cloud community. There were many of great seminars on everything from modern serverless architectures to more sophisticated DevOps approaches, but one session in particular stole the show for me.

The session was delivered by Yeshwanth L M on AWS Bedrock VPC Flow Log Analyser. This presentation was a breath of fresh air after far too many late nights staring at endless rows of black and white network logs. I made sure to snap some good pictures of the slides so I could deconstruct this great tool for you all.

The Network Log Nightmare

Let’s face it… sifting through raw network logs is a horrible, mind-numbing process.

Think of a Virtual Private Cloud (VPC) as a big, high-security office building. Each time a piece of data comes in, goes out of or moves across rooms, a digital security guard makes a line in a ledger. In the AWS ecosystem this ledger is called VPC Flow Logs. They record complete, rich IP traffic data across all your network interfaces, offering complete visibility into your network environment.

Does this sound familiar? If you’ve ever turned them on, you know what the catch is. The raw logs are quite verbose and complex immediately creating huge volumes of data.

To give you a feel for what this looks like in the wild, here’s an exact slice of the event data shown throughout the session. If you check your logs now, you will likely find a wall of text similar to this:

2 493062987015 eni-30076669 107.170.242.27 172.31.8.238 123 123 17 1 76 1433806982 1433807038 ACCEPT OK
2 493062987015 eni-30076669 172.31.8.238 107.170.242.27 123 123 17 1 76 1433806982 1433807038 ACCEPT OK
2 493062987015 eni-30076669 79.33.7.53 172.31.8.238 54517 23 6 3 180 1433807174 1433807218 REJECT OK
2 493062987015 eni-30076669 71.6.135.131 172.31.8.238 15314 21379 6 1 40 1433807224 1433807278 REJECT OK
2 493062987015 eni-30076669 172.31.8.238 108.61.56.35 123 123 17 1 76 1433807281 1433807338 ACCEPT OK

To get any useful information out of this digital alphabet soup , you need a specialised analysis . You need to be a wizard with advanced analytical tools or know complex query languages to even find out if your application is safe.

The Analysis Bottleneck

The logs are so extensive that development teams are soon faced with what speaker called the “Analysis Bottleneck.” This bottleneck impacts engineering teams in 4 big ways:

Time-Consuming Process: “Manual analysis is a big bottleneck, where it takes hours to dig through data that should be instantly available.”
Error-Prone Review: The immense amount and complexity of raw text may result in manual reviews subject to human error and oversight.
Skill & Accessibility Barrier: If a team member lacks in-depth technical knowledge of specific log querying systems, they are unable to access data.
Operational Inefficiency: These variables delay down incident response and hide crucial network security discoveries.

Why VPC Flow Logs Matter

You might be wondering...if they are such a headache to read, why don't we just avoid them?

But here's the thing... you can't turn them off if you care about your cloud infrastructure. They are of extreme importance for three main reasons:

Security Configuration: They’re critical for detecting security group and Network ACL misconfigurations that could expose your system to attacks.
Traffic Monitoring: They are essential for monitoring traffic patterns and detecting anomalies indicating potential security events or performance difficulties.
Troubleshooting: They are critical for diagnosing connectivity difficulties and improving your network security posture with data-driven insights.

Introducing the Bedrock-Powered Solution

In addition this is when it gets fascinating... What if you could chat with your network logs the same way you interact with a teammate on Slack?

That’s exactly what the Amazon Bedrock-Powered VPC Flow Log Analyser accomplishes. It entirely bypasses the typical log analysis headache via 3 fundamental pillars:

Gen AI Integration: It uses the latest generative AI from Amazon Bedrock to understand natural language questions with unmatched accuracy.
Plain English Queries: No complex query writing is required. You simply ask enquiries in simple English, like you would to a coworker.
Intelligent Translation: It automatically transforms your questions into accurate database queries to obtain useful insights from your flow logs.

Architectural Shifts: Traditional vs. Modern

To understand why this is a game changer, we need to understand the architecture progression. Think of your database as a huge library. If you want to know a specific information, the old technique is to study the Dewey Decimal System, locate the book, read the complete chapter.

In a Traditional Architecture, your VPC generates flow logs and pushes them to an Amazon S3 bucket. From there you need to configure Amazon Athena to analyse the logs, run custom SQL queries and tie it up to a visualisation dashboard. It works but needs effort, maintenance and a particular skill set to manage the queries.

Next, let’s look at the AWS Bedrock VPC Flow Log Analyser Architecture. The set-up is simply amazing. The VPC continues to push logs to S3 and CloudWatch. But instead of an engineer writing SQL queries by hand, the logs go straight into Amazon Bedrock. You just type into the system: “Is there any unusual activity from an unknown person?”

How the Data Processing Magic Works

So how can a raw string of text turn into an intelligent conversation? The tool orchestrates a smooth five-step Data Processing Flow:

Flow Log Retrieval: It takes the raw VPC Flow Log data directly from CloudWatch Logs.
Data Formatting: It converts those messy flow logs into a structured, highly readable format (e.g., 10.0.1.100:443 -> 10.0.2.200:80 (TCP) [ACCEPT] 1500B/10P).
Context Creation: It then produces a rather complete system prompt that includes summary statistics (total records, unique IPs, bytes, packets), the detailed records from the flow log, and explicit directions for analysing the data.
Bedrock Query: It summarises the prepared data and the user query, and sends it directly to the Claude 3 Sonnet model in Amazon Bedrock.
Response Processing: It takes the model’s output and sends the AI-generated analysis back to you.

Smart Token Management

Long story short… there are limits to how much text Large Language Models can process at once. Speaker also built a smart token optimisation layer to maintain the tool’s speed and efficiency:

Query Limits: It limits data to 150 flow log records maximum every query to prevent hitting token restrictions at any costs.
Max Tokens: It limits responses to 2000 tokens so that responses will be short.
Optimization: It is a small format which minimises token costs but retains the detail intact.

Absolutely, the complete lack of persistent resources is what makes this tool so elegant. It invokes the model directly. That implies no creation of AI agents, no storage of data within Bedrock, no permanent knowledge bases to maintain, and every query is 100% stateless.

Getting Practical: Prerequisites and Usage

The Prerequisites to execute this in your own environment are really straightforward:

AWS CLI configured with the right permissions.
Python 3.10 or higher.
AWS Bedrock access in your region.
VPC Flow Logs enabled (really the tool will guide you if they are not enabled)
IAM Permissions for: ec2:DescribeVpcs, ec2:DescribeFlowLogs, logs:FilterLogEvents, and bedrock:InvokeModel.

Once you clone the source and install the requirements, figuring out How to Use It is only four easy steps:

Run the Analyzer: Execute python vpc_flow_analyzer.py to start the tool.
Verify Flow Logs: Enter your VPC ID; the program will check if VPC Flow Logs are enabled and provide you some instructions.
Select Time Range: Specify the hours of data to analyze (like 1 hour, 6 hours, 24 hours, or 1 week).
Ask Questions: Query your flow log data using natural language.

Real-World Questions You Can Ask

The session showcased some great examples of questions you can ask, grouped by what you’re looking to discover:

IP & Port Analysis

"What source IP addresses do you see?"
"List all destination IP addresses"
"What destination ports are being accessed?"
"Which IP has the most traffic?"

Protocol & Security Analysis

"Show me all TCP connections"
"Which protocols are being used?"
"Which connections were rejected?"
"Show me suspicious activities"
"Are there any failed connection attempts?"

Traffic Analysis

"What's the largest data transfer?"
"Show me connections to external IPs"
"Which interface has the most traffic?"

Direct Invocation vs. Bedrock Agents

The presentation had a wonderful architectural comparison. Why would you use direct model invocation rather than develop a full Bedrock Agent?

The breakdown makes the option straightforward. Direct Model Invoke doesn’t require any setup, however Bedrock Agents require you to create an agent and build up a knowledge base. With direct invocation, no data is persisted, therefore data is always up to date from the flow logs. Bedrock Agents, in contrast, store data in knowledge bases and require constant data syncing.

When all is said and done, direct invocation offers a straightforward, stateless experience pulling live data directly from CloudWatch without complex maintenance.

Cost Considerations

The bottom line? This slim design keeps the money in your pocket. It avoids costly infrastructure, therefore the cost concerns are highly developer-friendly:

Pay-per-use: You are only charged for actual model invocations.
Input Tokens: The cost is only a function of the size of the flow log data plus the system prompt.
Output Tokens: Charges are based on response length (safely capped at max 2000 tokens).
No Setup Costs: There is absolutely no infrastructure or agent setup required.

Key Takeaways

Here’s a quick rundown on why this technology is such a giant leap forward for cloud teams:

Plain English is the New Query Language: You don’t need to be a SQL guru, or create complex Athena queries, to comprehend your network traffic. Just ask enquiries in a genuine manner.
Lightweight and Stateless: The tool gets live data immediately without persistent storage or complex setup by calling the model directly instead of heavy Bedrock Agents.
Highly Cost-Effective: You pay only for what you use. There are no knowledge bases to store, no agents to operate, hence your token expenses are very inexpensive.
Democratizes Security: In the end, this solution opens up the critical network data to the team, removing the skills barrier to debugging.

Conclusion

AWS Community Day Kochi 2025 was a fantastic example of how important local tech communities are. Yeshwanth’s breakthrough ideas reveal how generative artificial intelligence may remove all the friction points in ordinary cloud operations.

If you are developing a startup or running cloud deployments on a small budget, binding Amazon Bedrock straight to your VPC Flow Logs is a fast, stateless and simple solution to keep your network safe. Long story short… it makes a terrible operational duty into a really entertaining conversation.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi
Topic: AWS Bedrock VPC Flow Log Analyzer
Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

Build Your Own AI Support Assistant Using AWS Bedrock in Minutes

N Chandra Prakash Reddy — Tue, 14 Jul 2026 13:29:41 +0000

AWS Community Day Kochi on 20th December 2025 was a blast with awesome energy and great ideas. There were so many amazing lectures ranging from complex serverless designs to cutting-edge DevOps workflows. But one specific talk really stole the spotlight for me.

The talk was on how to construct a custom AI support assistant using Amazon Bedrock. I adore practical applications of AI. I watched the speaker take complicated generative AI and break it down into a simple architecture anyone could create in minutes.

We’ve all been there with a generic customer care chatbot that felt like a complete waste of time. You ask it a particular question regarding a corporate policy and it offers you a generic, unhelpful answer or just makes something up totally. Sounds familiar?

Here’s the deal: default large language models (LLMs) are super smart, but they don’t immediately know your confidential company data. This lecture was spot on for that topic, how to design a highly accurate context-aware support assistant without developing tonnes of complex code.

Understanding the Core Engine: Amazon Bedrock

Before getting into the architecture, the speaker focused on why Amazon Bedrock is such a game-changer for developers who want to build AI apps.

The Unified API Layer

Amazon Bedrock is a kind of universal travel adaptor for generative AI models. Different AI companies construct different LLMs, but Bedrock exposes a single, standard API to interface with them all.

Whether you want to use Claude, Llama, Mistral, Cohere or Amazon’s own Titan models, the request process is the same. If you wish to move between models, just update the modelId parameter in your settings and everything else stays exactly the same.

This means you don’t have to waste time learning a whole specialised API every time a provider delivers a new model upgrade. It enables it extremely smooth to explore, compare performance and upgrade your backend.

The session explored two essential APIs that power all this interaction:

InvokeModel: You can use this API to provide direct text prompts to any text model hosted on the platform.
InvokeAgent: This API provides direct access to higher-level intelligent agents that orchestrate complex workflows.

The Secret Sauce: Why Context is Everything

To understand why we need things like Bedrock, we need to understand how LLMs work. The speaker gave a great real-world example of a normal firm Earned Leave (EL) policy to highlight the huge difference context makes.

The Earned Leave Dilemma

Now imagine you ask a normal out of the box LLM a simple question: "What is the earned leave policy?

The model has no company-specific data, therefore it can only offer a textbook explanation of what earned leave normally entails. It can say you get 1-2 days a month or that you always need management clearance. It is correct as a general notion , technically , but totally worthless for an employee who has to know the particular rules in his organization .

Asking with Relevant Context

Now think that you ask the identical inquiry, but this time you give the model a sample of your real corporate HR document along with the question. The context indicates that permanent employees become eligible for earned leave after completing 6 months of employment, and that they earn 1 day of leave for every month worked.

The model has that data there in front of it, so it can instantly deliver a hyper-accurate, personalised solution suited only to your organization.

Enter RAG: Retrieval-Augmented Generation

It is not practical to manually provide this context for every single user inquiry, and that is where Retrieval-Augmented Generation (RAG) might help.

RAG is the AI model version of an open-book test. Rather of the model guessing an answer based on its training memory, RAG lets the model browse real, proven information from your documentation before answering a user.

Speaker has broken down the RAG workflow into three simple, easily understood stages:

Retrieve: The algorithm then searches your private papers, PDFs, FAQs, Amazon S3 buckets, or Confluence pages for information relating to the user’s question.
Augment: The system then takes that obtained content and automatically puts it as a temporary context layer directly adjacent to the user’s initial prompt.
Generate: The LLM reads both its baseline knowledge and the new context you gave to generate a very accurate response based solely on your data facts.

The Building Blocks of a Support Assistant

In the past, you would have to build a full RAG system from scratch, which meant managing separate vector databases, creating data pipeline crawlers, and text chunking logic. Here’s where things get interesting: Amazon Bedrock has three built-in components that do all this heavy lifting natively.

1. Bedrock Knowledge Bases

A Knowledge Base provides foundation models and agents with immediate access to your company's exclusive data sources. There are three easy steps to setting one up:

Connect your data source: Point Bedrock at your raw files sitting in Amazon S3, Confluence, SharePoint or even a Web Crawler.
Choose an embedding model: Select a model like Titan Embeddings V2 to automatically turn your written documents into mathematical vectors that machines can search instantaneously.
Select a vector store: Store the vectors in a separate database. With Bedrock this is simplified by having a built-in, fully managed integration with Amazon OpenSearch Serverless.

2. Bedrock Agents

An LLM can only chat back and forth, and can’t do operational activities without an agent. Agents are an intelligent orchestrator for your application.

They enable the LLM to automatically perform lookups on your Knowledge Base, safely access external APIs or development tools, conduct complicated multi-step workflows, and keep track of the ongoing conversation state so that the user never needs to repeat himself.

3. Bedrock Guardrails

What if a user tries to deceive your support assistant into revealing important internal code? Or asks it improper questions?

Guardrails provide a layer of protective policy immediately on top of your models, independent of the underlying LLM you choose to use. They give you actual operational controls including:

Safety Filters: Automatically block harmful, toxic, or completely unacceptable content.
PII Protection: Identify and hide sensitive personal information such as phone numbers, addresses, or emails before processing.
Topic & Response Control: Restrict the bot to topics that are company-approved and enforce organisational response policies.

Architecture Deep Dive: How It All Fits Together

In the session the speaker walked through the overall architecture diagram of how various components interact in real time.

When a user submits a question using a customer support app, the question is sent immediately to the Bedrock Agent. The agent immediately queries the OpenSearch Service Vector Store for relevant document context.

Your raw documentation stored inside an S3 bucket is fed through an embedding model and regularly updated inside that exact same vector store behind the scenes.

The agent then combines the user’s initial inquiry with the newly retrieved context snippets to create a very robust, augmented prompt. This entire prompt is sent directly to the target LLM. The model examines the document context, formulates a precise answer and feeds it back through the agent to give an exact response straight to the user.

Putting It Into Practice: The Console Demo

To demonstrate how easy this is the speaker guided us through a console recording of a live setup, showing the precise method to create it.

We saw the creation of a demo knowledge base entitled mrinal-knowledge-base-bedrock-demo. In the AWS Management Console, you get a simple, step-by-step wizard that:

Provide Knowledge Base details: Enter your custom name and an optional description to keep your resources organized.
Configure data source: Link your target storage system where your documentation lives.
Configure data storage and processing: Select your embedding model and vector database settings.
Review and create: Confirm your architecture settings and hit deploy.

You can even choose for the console to automatically build and assign a new Identity and Access Management (IAM) service role. This guarantees your components have precisely the security permissions needed to communicate properly without you having to manually develop specific JSON IAM policies.

Taking It Further: Next Steps for Production

The fundamental RAG engine is super quick to build in the console, but the speaker reminded us that the real power is in where you connect it next.

Once your Bedrock Agent and Knowledge Base are up and running, you can quickly expand its capabilities to your exact business stack:

Integrate with chat platforms: Connect your agent directly to your daily communication platforms, such as Slack or Microsoft Teams, so your team may query internal documents in real time.
Enable email-based support: Integrate your agent with Amazon Simple Email Service (SES) or Amazon WorkMail to automatically parse incoming customer emails and produce precise contextual answers.
Connect incident management tools: Let your agent directly talk to operations systems (PagerDuty, Jira, ServiceNow, etc) to dynamically open support logs, or auto-triage engineering requests.

Key Takeaways

Sitting during this session, it struck me how fast the barriers to entry for developing powerful AI are coming down. If you aim to develop your own helper, remember these fundamental lessons:

The standard API saves major time: You don’t need to rewrite the architecture of your application just because a new model launches. Bedrock does the heavy lifting, so you can swap models in an instant by altering one parameter.
Context solves the hallucination problem: LLMs not trained on specific data have an ability to guess out of the box. Grounding the model with a precise documentation context via a RAG pipeline to maintain it correct and relevant.
Infrastructure management is optional: Instead, with fully integrated solutions like OpenSearch Serverless and managed Bedrock Agents, you can avoid weeks of effort in setting up infrastructure and implementing customised orchestration scripts.
Security should never be an afterthought: Built-in solutions like Bedrock Guardrails allow you to safely deploy user-facing AI by hiding PII data and enforcing strict subject boundaries from the start.

Conclusion

AWS Community Day Kochi was a great reminder of how practical and accessible cloud technology is today. To summarise, you don’t need a big team of data scientists or millions of rows of specific training data to construct a smart, business-aware AI assistant.

With Bedrock Knowledge Bases, Agents, and Guardrails, you can go from raw PDF documents to a fully operational, safe, and context-aware support bot in minutes. The tools are production ready, so now it’s time to go out and build.

If you have been waiting to build your own AI tool because the environment felt too complex, I highly encourage diving into the AWS interface and trying Bedrock for yourself. It’s highly developer friendly, ridiculously quick to get up and running, and powerful enough to revolutionise how your team does operations.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: Build Your Own AI Support Assistant Using AWS Bedrock in Minutes

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

Scaling Custom LLMs on EKS: Trainium and Inferentia2-Powered AI Infrastructure

N Chandra Prakash Reddy — Mon, 13 Jul 2026 15:19:24 +0000

I attended AWS Community Day Kochi on 20th December 2025 and it was full of amazing insights. There were a lot of great seminars throughout the day on different AWS services but one session in particular totally caught my eye. The session featured a technical deep dive, “Scaling Custom LLMs on EKS: Trainium and Inferentia2-Powered AI Infrastructure”.

Here’s the rundown of what was covered and how it all works and why it matters if you’re building AI apps today.

The Challenge: When Do You Outgrow Standard APIs?

Let's face it, using off-the-shelf APIs like OpenAI or Anthropic is great for rapid prototyping. You obtain an API key, you write three lines of code, and you have intelligence embedded right into your program. But what if your app goes viral and takes off?

The speaker hit us with a hard truth straight away: API costs increase dramatically with request volume. Think of it as a cab. It is quite useful for short, occasional journeys. However, if you are driving 100 miles each day, then owning your own automobile makes a lot more financial sense.

Session data suggests the important inflection point is at around 10 million requests each day. That’s when third party APIs become too expensive and building a custom platform is a very practical, affordable necessity.

Enter AWS Purpose-Built Chips

This is a huge scalability problem. To overcome this AWS has come up with purpose-made hardware accelerators that are specifically developed and constructed from the ground up for Generative AI workloads.

We all know what a typical GPU is, but they are expensive to rent and hard to get at scale. AWS has created its own silicon to solve this very problem.

AWS Inferentia (Inf1): It offers the lowest cost per inference in the cloud for Deep Learning models, with costs up to 70% cheaper than normal EC2 instances.
AWS Inferentia2 (Inf2): Specifically designed for large language model (LLM) and diffusion model. It offers up to 40% better price performance than equivalent EC2 instances.
AWS Trainium (Trn1): Your powerhouse for model training. This means you can achieve up to 50% savings on training costs compared to equivalent Amazon EC2 instances.

The Architecture: Bringing it All Together on EKS

Great software is essential to make hardware useful. This is where the AWS Generative AI stack and SDK for Neurone come into play. Neurone SDK bridges third-party applications such as PyTorch, Ray and vLLM to the underlying Trainium and Inferentia hardware.

The worldwide plan showed a very resilient arrangement across regions. Users connect to an Application Load Balancer via a Global Accelerator, and the Application Load Balancer sends the traffic to a Karpenter-managed, dynamically provisioned Amazon EKS cluster. The models are directly served from the hardware nodes using an Nvidia Triton Inference Server in the cluster, running vLLM and the Neurone ML SDK.

The Reality of Training and Serving at Scale

Imagine painting a big canvas all by yourself with a teeny little brush. That's about how it feels training a massive LLM on a single GPU.

The speaker states that training a certain large model on a single GPU may take ~120 days and cost $500k+. A Trainium Cluster provides the ability to run numerous processors in simultaneously. That takes overall training time down to just 14 days, and cuts the cost to $150k - an astounding 70% cheaper.

Just as vital is serving. 500ms latency is a sluggish experience for traditional GPUs. Inferentia2 lowers this down to a 100-200ms latency, making the experience feel nearly quick, all while being 70% cheaper.

Hands-On: The EKS Configurations and Code

This is where it gets good. The speaker dived straight into the live demo, presenting the real setups that make this magic happen.

1. The Kubernetes Magic Line

If you want to use a Trainium or Inferentia device in your Kubernetes deployment, just add one magic line in your resource limits: [aws.amazon.com/neuron](https://aws.amazon.com/neuron): "1" .

Here is what the manifest looks like:

# The Magic Line - Requesting Neuron Device
resources:
  requests:
    cpu: "4"
    memory: 16Gi
    aws.amazon.com/neuron: "1"   # <- Request 1 Trainium/Inferentia device
  limits:
    cpu: "8"
    memory: 32Gi
    aws.amazon.com/neuron: "1"

To bridge the gap between Kubernetes and the physical hardware, you deploy a DaemonSet called the neuron-device-plugin. It automatically discovers the hardware and mounts the /dev/neuron* paths:

# DaemonSet discovers and exposes Neuron devices
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: neuron-device-plugin
  namespace: kube-system
spec:
  template:
    spec:
      containers:
        - name: neuron-device-plugin
          image: public.ecr.aws/neuron/neuron-device-plugin:2.19.16.0
          securityContext:
            privileged: true  # Required to access /dev/neuron*
          volumeMounts:
            - name: device-plugin
              mountPath: /var/lib/kubelet/device-plugins
            - name: neuron-dir
              mountPath: /dev  # Access Neuron devices

2. Protecting Your Budget with Taints

Here’s a huge money saving tip. Trainium on-demand nodes cost around $1.34 per hour. You do not want to run simple system pods like CoreDNS on those pricey ML instances!

Using Terraform, you can apply a taint to block non-ML pods:

# Terraform: Node Group with Taint
taint {
  key    = "aws.amazon.com/neuron"
  value  = "true"
  effect = "NO_SCHEDULE" # Block non-ML pods!
}

Then you just add a toleration in your kubernetes deployment so your ml pods can schedule there:

# Kubernetes: Pod must tolerate to run on Neuron node
tolerations:
  - key: "aws.amazon.com/neuron"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"

nodeSelector:
  node-type: trainium          # Target Trainium nodes
  aws.amazon.com/neuron: "true"

3. Training with PyTorch + Neuron

The real training code with Pytorch and Hugging Face is delightfully simple. Notice how loading the model in bfloat16 offers you a 2x memory save right away:

# Training on Trainium
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model
model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    torch_dtype=torch.bfloat16,  # BF16 = 2x memory savings
)

# Assume model_path is defined
tokenizer = AutoTokenizer.from_pretrained(model_path)
tokenizer.pad_token = tokenizer.eos_token

# Simple training loop
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-5)
model.train()

for step, batch in enumerate(dataloader):
    outputs = model(
        input_ids=batch["input_ids"],
        attention_mask=batch["attention_mask"],
        labels=batch["labels"],
    )
    loss = outputs.loss
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

    print(f"Step {step} - Loss: {loss.item():.4f}")

4. Serving the Model with FastAPI

The speaker exhibited a clean FastAPI wrapper providing the trained model on Inferentia2.

# Inference Server on Inferentia2
from fastapi import FastAPI
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

app = FastAPI(title="TinyLlama Inference API")

# Load model once at startup
model = AutoModelForCausalLM.from_pretrained("/models/tinyllama")
tokenizer = AutoTokenizer.from_pretrained("/models/tinyllama")

@app.get("/health")
async def health():
    return {"status": "healthy"}

@app.post("/v1/generate")
async def generate(prompt: str, max_tokens: int = 100):
    inputs = tokenizer(prompt, return_tensors="pt")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_tokens,
            temperature=0.7,
            do_sample=True,
        )

    return {
        "generated_text": tokenizer.decode(outputs[0], skip_special_tokens=True)
    }

The Bottom Line: Math and Money

Your workload is the single most important factor for your memory needs. For a 7B parameter model running in BF16:

Inference Memory: Needs about 16.8 GB of memory. This fits nicely on an inf2.xlarge instance.
Training Memory: You need a big 56 GB because you need to save gradients and optimisers.

The final pricing comparison was the most jaw-dropping slide. APIs with significant traffic can easily run to $60,000/month off-the-shelf. Running your own EKS cluster with Inf2 On-Demand instances reduces that to $547.

But the true game-changer is that you are able to set up Spot Instances with Terraform.

# Terraform: Enable Spot Instances
resource "aws_eks_node_group" "trainium" {
  instance_types = ["trn1.2xlarge"]

  # THE MONEY SAVER 
  capacity_type = "SPOT"  # vs "ON_DEMAND"

  # Scale to zero when not training
  scaling_config {
    desired_size = 0      # Start at zero!
    min_size     = 0      # Allow scale to zero
    max_size     = 1
  }
}

EKS with Inf2 Spot instances (which can scale down to zero when idle) brings the monthly cost down to a mere $166. That's a crazy inexpensive $0.17 every 1K inferences!

Key Takeaways

The 10M Threshold: Third party LLM APIs stop making economical sense and specialised infrastructure becomes needed after 10 million daily requests.
Massive Cost Slashes: Move to an EKS cluster with Trainium and Inferentia2 and cut 50% of your training costs and 40% of your inference costs.
Budget Guardrails: Kubernetes standards and tolerations prohibit basic system pods from schedule-squatting on your premium $1.34/hr ML nodes.
Performance Without Compromise: Lowering infrastructure expenses doesn’t mean losing user experience. Inferentia2 readily maintains a latency of 100-200ms and a sub-100ms P99 latency even at a large 70B parameter scale.

Conclusion

Building custom Generative AI infrastructure is no longer some black art for the tech giants with endless cash at the end of the day. AWS has democratised the hardware layer with purpose-built silicon, and the software gap is bridged with open-source tools like the Neurone SDK, making it developer-friendly.

Long story short, if you are establishing an AI firm or trying to extend an engineering staff beyond basic API wrappers, it's well worth the effort to break free from third-party locks by understanding this purpose-built AWS stack.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: Scaling Custom LLMs on EKS: Trainium and Inferentia2-Powered AI Infrastructure

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

From Lake to LLM: Building AI-Ready Data with Amazon S3 Tables

N Chandra Prakash Reddy — Sun, 05 Jul 2026 16:56:04 +0000

Participating in the AWS Community Day Kochi on December 20, 2025 was an absolutely fantastic experience. There is always a certain type of energy being surrounded by passionate developers, cloud architects and techies. There were so many amazing seminars throughout the day but as someone who is really interested in data architecture, one particular tech session immediately grabbed my attention.

The title of the talk was “From Lake to LLM: Building AI-Ready Data with Amazon S3 Tables”. Let's be honest, getting your data truly ready for Artificial Intelligence is typically a major hassle. We hear all this excitement about Generative AI, but very few people talk about the filthy plumbing it takes to make it work. It was a breath of fresh air of a session since it handled that precise plumbing issue head-on.

Here's my thorough dive into what I learnt, including the session's insights and a few comments of my own to help break down the more complex aspects.

The Messy Reality of Today’s Data Lakes

The speaker opened the discussion by tackling the elephant in the room: the enormous “Enterprise AI Adoption Gap”. Does this sound familiar? Many of us have encountered this first hand while attempting to construct machine learning models for our firms.

The main concern uncovered in the session is that bad data in S3 is a real blocker to AI adoption. Think of your company's database as a giant public library. If books ( your data ) are just scattered randomly on the floor instead of being carefully sorted on labeled shelves , nobody can find what they need . This is exactly what is happening with traditional data lakes with scattered files and profound inconsistencies and absolute chaos.

What is Actually Blocking AI?

You might be thinking, why can’t we just point a Large Language Model (LLM) at our existing data lake and let it sort things out. The talk nicely outlined the technical blockers:

Raw S3 Lacks Structure: Basic, raw S3 storage has no built-in schema and query semantics. It only stores files, it doesn't know what is in them.
Siloed Workloads: Usually Business Intelligence (BI) teams and AI teams work with completely different storage pathways. This implies you’re paying to duplicate data and those two copies eventually get out of sync.
Data and Schema Drift: Over time, your data format will vary (this is termed "drift"). The slides made clear that schema drift often disrupts downstream pipelines. Data drift also leads directly to wildly uneven AI outcomes.
Lack of Versioning: A simple S3 setup means losing transactional assurances and tight versioning, which makes your machine learning models much less reliable.
The RAG Headache: Implementing Retrieval-Augmented Generation (RAG), which is how you allow an AI search your private papers, often puts large, highly advanced components into your system.

The point is, LLMs are not magic. They really need to have clean, consistent, and highly controlled data in order to work correctly. Feed them rubbish and they'll hallucinate poor data.

The Fix: A Unified AI-Ready Data Foundation

Enterprises are desperate for a uniform data foundation to fix this chaos. We require a single platform that can run both typical SQL queries and advanced analytics workloads, instead of duct-taping different services together.

The speaker stressed that this modern base needs to have native support built-in for RAG, vector embeddings and seamless LLM-driven insights. It needs centralized governance, accurate data lineage (understanding where your data comes from), and reproducibility across all different workloads.

Entering the Modern Lakehouse Approach

This is when things become interesting. The session continued with the “Modern Lakehouse Approach”. In particular, they un-veiled the capabilities of Amazon S3 Tables that natively build on top of the open source Apache Iceberg format under the hood.

That means S3 Tables deliver the ACID transactional guarantees right to your data lake. An ACID transaction is like sending money electronically. When you transfer ₹1000 to a friend, the system ensures that the money is deducted from you and added to your buddy's account concurrently. If the internet goes down mid way the entire transaction is cancelled. It never leaves money floating about in cyberspace. S3 Tables provides your data files with the same unassailable reliability.

This service provides queryable table semantics and strong schema and metadata consistency. It creates strong governance at the data layer itself, providing the all-important uniform base for both analytics and AI.

Structuring the Chaos: The Medallion Architecture

So how do you manage this new strong lakehouse? The speaker was very bullish on the “Medallion Approach” in data engineering. Imagine this as filtering drinking water. You begin with a muddy river, then you run the water through coarse filters, then fine filters. At the end, you have pure and safe bottled water.

Bronze Layer (The River): This is where raw ingestion and historical data are landed straight from streaming sources such as Kafka and Kinesis or batch sources such as Apache Spark and regular CSV/JSON/TXT files
Silver Layer (The Filter): This is where the raw data is thoroughly screened, cleansed and enriched. Null values are dropped, formats are standardized.
Gold Layer (The Bottled Water): Finally the data is converted to company level aggregates. This is the clean, high-quality data that executives and AI models are fed.

This layered pipeline is built on a solid base of data quality and governance, and flows directly into streaming analytics, BI reporting, data science/ML environments, and data sharing platforms.

Seeing it in Action: Sales Analytics Architecture

To be fair, abstract architectural patterns can seem a little frightening. But the presenter simplified it down with a very practical use case: Building a Sales Analytics platform with Customer Feedback.

The architecture diagram showed a beautiful, logical flow:

Data flows from a source S3 bucket via automated ingestion jobs into the Bronze S3 table.
Transformation jobs then move it into the Silver S3 table.
A important step between the Silver and Gold layers is the production of text embeddings. This is the part where client text reviews are converted into numbers so the AI can understand them.
The refined data lands in the Gold layer S3 table.

From that Gold layer, the data is spread out. Connects to SageMaker Unified Studio for large ML workloads. It also interfaces with a Conversational Chat Interface that runs on Anthropic's Claude LLM and a Model Context Protocol (MCP) Server.

Claude and S3 Tables: The Ultimate Chat Interface

The live demo of the MCP Server querying S3 Tables with Claude was definitely the highlight of the session. The demonstration typed a request into Claude asking it to explain the revenue pattern for EMEA in Q2 2024 and to summarize customer comments connected to that.

Claude logged easily into the database and looked over the results. It said total Q2 revenue was $840,000, but quickly noticed a huge 33% revenue loss for “Product A (Enterprise).”

Instead of a human data analyst hunting for the reason, Claude cross-referenced the customer input. It brought to light the instability and app crashes at peak hours of Product A, leaving the team frustrated. It also brought up criticism for “Product B (MidMarket)” about a competitive gap where customers may churn and “Product C (SMB)” for unclear billing and invoicing problems.

In brief, this design makes complex, large database tables instantly usable business insight, simply by conversing with it in simple English.

Why This Makes Your Data "AI-Ready"

The end result? Amazon S3 Tables are meant to be AI-ready.

They give dependable transactions that result in consistent data pipelines. Your AI is not learning from incomplete files.
They allow flexible schema evolution, so you can evolve easily to changing machine learning models without damaging downstream systems.
They actively address the schema drift and lack of versioning that has typically restricted ML reliability.

S3 Tables solve these fundamental infrastructure issues, and deliver the clean, consistent and regulated datasets that LLMs demand.

Enterprise Gains (The Payoff)

Ultimately, embracing an AI-Ready Lakehouse is more than a technological flex, it will mean huge, measurable organizational benefits.

This architecture will allow organizations to view:

Significantly faster delivery of AI and analytics solutions to their customers.
Far more reliable and consistent LLM-generated outputs (reduced hallucination!).
Reduced infrastructure complexity and overall cloud costs.
Better compliance with stringent compliance and corporate governance norms.
Scalable performance specifically designed to handle rapidly growing datasets.

Key Takeaways

If you’re skimming, here’s a fast summary of the most important things you need to know from the session:

Raw storage is not enough: Just throwing files into an S3 bucket results in schema drift and inconsistencies. LLMs require clean, regulated datasets to produce reliable outputs without hallucinations.
Transactions matter: S3 Tables (powered by Apache Iceberg) provides ACID transactional guarantees to your data lake, preventing broken pipelines so your models are training on reliable data.
Layer your data: The Medallion Approach (Bronze, Silver, Gold) is the best technique to turn messy, raw intake into high quality, enterprise level aggregates appropriate for AI consumption.
Unify your foundation: You don’t have to buy and operate different storage routes for regular BI reporting and advanced AI workloads. A modern lakehouse can manage SQL, analytics and LLM-driven insights under one roof.

Conclusion

At the end of the day, your generative AI applications are only as good as the data you provide them. The session at AWS Community Day Kochi was a reminder that the conventional, chaotic data lake is no longer good enough for modern needs.

With a modern lakehouse strategy using Amazon S3 Tables, we can finally simplify infrastructure complexity and generate much higher trust in our LLM outputs. It takes a little planning to get the right design but the scalable performance and strong oversight you obtain is totally worth it. In short, it’s time to improve our data foundations from a storage lake to a real AI engine.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: From Lake to LLM: Building AI-Ready Data with Amazon S3 Tables

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

14x Cheaper AI: A Real-World LLM Distillation Case Study on Bedrock

N Chandra Prakash Reddy — Sun, 05 Jul 2026 13:06:31 +0000

On 20 December 2025, I was fortunate to be a part of the AWS Community Day Kochi. There were fantastic sessions going on throughout the event, but there was one presentation in particular that caught my whole attention and wouldn’t let go.

The speaker came on stage and delivered a big result right out of the gate – they reduced their AI operational costs by 14x using AWS Bedrock. But this wasn’t a highlight reel of quick success. Let’s face it, tech talks that only display the wonderful stuff don’t educate us much. Instead, there was a clear story of how the squad failed again and again on the route to the big victory.

This session was pure gold if you are a developer, or a firm seeking to scale AI features without burning through your entire runway. Here’s a look at the trip, the technological challenges, and how they finally cracked the puzzle.

The Business Problem: A 3-Body Challenge

But before we get to the solution, we need to understand the nightmare the team was dealing with. They named it the “3-Body Challenge.”

The trouble is...they were drowning in data. Specifically, they were being overwhelmed with unstructured communications about cargo bookings.

The emails were bilingual and consisted of a crazy mix of Japanese and English content.
They needed their system to be able to correctly extract 23 complicated entities from these emails, such as Air Waybill (AWB) numbers, Flight Numbers, Weights and Dimensions.

The Accuracy vs. Cost Dilemma

The aim was to carry out real-time automatic Named Entity Recognition (NER). The system needed to be low latency, and have a very high accuracy rate of over 95% (f1 score), to be useful in the production pipeline.

You might be wondering why not just throw a Large Language Model (LLM) at it? They did well. And the LLM readily met the precision required. But the operating cost at that high volume was a deal breaker.

Sound familiar? This is a trap many teams get into. They had designed a system that worked well, but they knew they could never build a business around a 14x cost problem.

Rethinking the Core Problem

The first important change in their thinking was in how they approached Named Entity Recognition. Instead than using typical BIO (Beginning, Inside, Outside) tagging, they defined NER as a Sequence-to-Sequence (Seq2Seq) task.

Generating Structured JSON

To grasp this, picture it like an e-commerce checkout system. You don’t want the system to just highlight random goods in a shopping cart, you want it to create a well-formed receipt.

The input (sequence 1) in their case was the raw, jumbled email text prompt asking the model to extract all 23 entities as a JSON array. The expected output (sequence 2) was the exact JSON text produced that matched those entities.

The Technical Goal: Knowledge Distillation

To achieve the high accuracy of a big LLM without the massive price, they turned to a concept known as Knowledge Distillation.

Think of your database as a huge library and the “Teacher Model” as the chief librarian who has read and comprehended every book. The teacher is large, complex and expensive to consult. The purpose of distillation is to compress the knowledge and transfer it to a “Student Model”. The student is smaller, considerably faster and much cheaper to run, offering you the best of both worlds, great precision and low cost.

Evaluating the Distillation Options

The speaker outlined the major routes he may take to achieve this knowledge transfer:

Option 1: Logit-Based (e.g., DistilBERT): This method uses a metric called KL Divergence to match the student's final output probabilities (logits) to the teacher's. It is easy, fast and effective. But it typically misses a lot of the sophisticated internal “reasoning” of the teacher model.
Option 2: Feature-Based (e.g., TinyBERT): That is, to align the internal hidden states and attention mappings of the two models. these transfer knowledge really deep. The negatives? It's quite brittle. It requires model architectures to be same and is quite sensitive to throwing shape_mismatch errors.
Option 3: Token-Based: Here, the teacher model is compared with the final output sequence of the pupil token by token. It learns from the teacher's soft labels and is suitable for generative Seq2Seq jobs such as the JSON extraction they needed.

They decided to go with the Token-Based method as they were generating JSON arrays. And now it gets interesting, and by fascinating I mean extremely frustrating for their engineering team.

The Engineering Nightmares

Attempt 1: The Token Mismatch Wall

Their initial approach was a custom token distillation built with PyTorch. They sought to distill the Llama 3-8B model to the Llama 3-1B model with their specialized Seq2Seq task.

In fairness, the logic was reasonable, but the technological reality was a failure. For token-based distillation, you need absolute, perfect token alignment in order to effectively distill that output JSON. They ran into a big problem: the Llama 3 tokenizer and their Japanese/English bilingual text were misaligned.

They were trying to compare output sequences that just didn’t line up, and the training loss got wildly unstable. This caused a constant token_mismatch_error.

Attempt 2: The Brittle Architecture Wall

Try 2. No way they were giving up. They tried to run a Logit/Feature-Based distillation technique with a library named TextBrewer.

This, too, died a technical death, as the solution was too brittle and required completely matching architectures. The library was quite strict about requirements and their specific Llama models were incompatible.

The operation failed again, generating a shape_mismatch_error. The team found that they were spending 100% of their time fighting engineering difficulties and 0% of their time on true data research.

The Pivot: Isolating the Real Problem

The team stepped back and saw their problem was not a terrible theory. Their problem was bad engineering. They were getting beaten on the two hardest segments of the pipeline:

Tokenizer and Architecture Alignment.
Setting up stable, distributed training environments.

In short, they required a completely managed solution to do the heavy lifting on the engineering requirements things like framework selection, dataset preparation, and normalizations so they could focus entirely on addressing their business challenge.

Enter Amazon Bedrock Model Distillation

They migrated their whole pipeline to Amazon Bedrock. The new method looked like this:

Take the user prompts and feed them into the large Teacher model.
Generate high-quality synthetic data.
Use that data to train the smaller Student model and transfer the knowledge.
Deploy the customized distilled model for real-world inference.

The workflow was pleasantly simple. The prepared prompt dataset was just taken by a data scientist and turned into a JSONL file and uploaded to an Amazon S3 bucket. They then picked whatever instructor and student models they wanted from within the Amazon Bedrock service to start the distillation job.

What made this one work while all the others didn't? It wasn’t magic, the speaker pointed out. I checked the CloudWatch training logs and it just... worked.

It solved alignment: By using the compatible Nova Pro and Nova Lite model family, there was zero token mismatch.
It solved stability: The managed service handled all the complex orchestration behind the scenes.

Their training loss went from 0.05 to a very accurate 0.008 in just 4 epochs and 70 total steps.

The Results: Fast, Accurate, and Cheap

They ran the newly distilled Nova Lite model that took the jumbled input prompt and produced a perfectly formatted JSON output array of the desired entities.

The bottom line? The stats tell the story:

The Teacher (Nova): Achieved a 97% overall F1 score (96.3% English, 95.4% Japanese) but cost 14x more to run.
The Student (Nova Lite): Achieved a 95.085% overall F1 score (96.535% English, 93.635% Japanese) at the 1x baseline cost.

They hit their >95% accuracy goal while entirely eliminating the 14x operational cost overhead!.

A Deeper Look at the Errors

Always seeking improvement, the team took a further look at the little 1.9% accuracy disparity between teacher and student.

Language: The student did significantly worse on the Japanese text (93.6% vs 95.4% for the teacher). Their next immediate step to fill the gap is to extend and increase the modest 150-sample Japanese dataset.
Complexity: Approximately 20% of the student’s errors were in the “long-tail” or multi-part entities. For example elaborate layered instructions such as “Fragile; refrigerate below 4°C” In these tough edge instances, the teacher model’s richer baseline reasoning nevertheless came out on top.

Key Takeaways

Generative NER is a game-changer: By moving from typical BIO tagging to a Seq2Seq technique, you gain amazing flexibility when you need to handle complex multi-entity extractions.
Your task dictates your method: If you pick a Seq2Seq task you are forced in a Token-Based distillation technique. Just be ready for the brittle engineering and alignment needs that are part of it.
The real win is cost, not speed: The big 14x improvement was about operational savings. Both models had similar inference performance but the smaller Nova Lite reduced the hefty financial burden of provisioning large LLM throughput.
Offload the engineering: If your team spends 100% of its time resolving architecture incompatibilities, they’re not conducting data science. Get rid of those inflexible tokenizer bottlenecks altogether with managed services like AWS Bedrock.

Conclusion

Sitting in the audience in Kochi at the conclusion of the day, I was reminded that the road to a massive technical win is seldom a straight line. The team didn’t just build a 14x cheaper AI, they failed forward, knew when they were in an engineering trap, and pivoted to a managed solution that allowed them to focus on solving their underlying business challenge.

If you’re producing AI solutions, don’t be scared to change your architecture when operational costs start to threaten your business model. At times, the smartest technological option you can make is simply to let a managed service do the heavy lifting.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: 14x Cheaper AI: A Real-World LLM Distillation Case Study on Bedrock

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

TDE Inside Out - Protecting SQL server data at rest on AWS

N Chandra Prakash Reddy — Sun, 28 Jun 2026 15:44:20 +0000

It was an awesome day at AWS User Group Chennai Meetup on 15th November, 2025. There is always something special about getting together with the local community, sharing real-world stories, and geeking out over cloud architecture. There were several interesting lectures throughout the day on a wide variety of topics including serverless and containers, but one particular talk on database security particularly got my attention.

Database security can seem complex, especially with enterprise workloads, but this webinar did a great job of breaking it down. Let’s lay down exactly how you can protect your Microsoft SQL Server data at rest on AWS without breaking a sweat.

The Core Challenge of Data at Rest

Let’s face it… we spend a lot of effort on network perimeter security. We set up firewalls, maintain security groups, and ensure that all data in transit is encrypted using TLS. But what if someone gets hold of your underlying storage backup or a raw database snapshot?

Do you know this? Security is frequently an afterthought until there is an audit or breach. When your raw database files are stored on the storage layer unencrypted, anyone with access to those files can read your sensitive data. This is where the concept of securing data “at rest” is non-negotiable.

Imagine your database is a bank with strict security. Locking the main door and checking IDs at the entrance, is like protecting data in transit. But if the cash is just lying around on open tables inside the vault, then the whole world wins when they sneak past the front door. Protecting data at rest is like putting that cash in separate, reinforced lockboxes inside the vault.

What Exactly is Transparent Data Encryption (TDE)?

The Basics of I/O Level Protection

That’s where Transparent Data Encryption, or TDE, comes in. TDE is a capability that is incorporated into Microsoft SQL Server. It encrypts the real files that are written to disk to protect your data. The beauty of TDE is in the name, it is totally transparent to your application.

You might be thinking, what does “transparent” look like in practice. In other words, you do not need to rewrite your application code, update your SQL queries or change your database schemas. SQL Server does all the hard work for you, behind the scenes.

SQL Server receives data from disk, decrypts it on the fly, and puts it into memory when your application requires data. As data is refreshed back to the storage layer, SQL Server encrypts the data as it hits the disk. This procedure is completely I/O based . It is intended for page level encryption .

What Gets Protected?

When you turn on TDE on a database it’s not just the major tables that are encrypted. This safeguards the whole database ecology on that instance. TDE actively protects the following key elements from session information:

Data Files (.mdf): The main storage files where your actual tables, indexes and user data reside.
Log Files (.ldf): The transaction logs that record all the changes made to the database, and are critical for recovery.
Backup Files (.bak): Any native backups of databases that you produce are automatically encrypted at the time of creation.
TempDB: The system database for temporary items, internal joins and sorting.

Unpacking the Encryption Hierarchy

The Chain of Trust

Here’s where things gets interesting. TDE does not depend on a single password or a single key for the security of your data. Instead, it uses a complex, multi-tiered encryption system sometimes referred to as a chain of trust.

This hierarchy is like a classic Russian nesting doll. You have to unlock a bunch of bigger dolls to get to the tiny doll inside (your real data). If one layer is broken or missing, the entire chain breaks, maintaining the security of your data.

Breaking Down the Layers

So let's walk through how these keys work together from the top down to develop a strong security model:

The Service Master Key (SMK): This is the root of the entire encryption hierarchy . It is automatically created at the very top level when the SQL Server instance is first setup, and is encrypted by the underlying Windows operating system or AWS infrastructure.
The Database Master Key (DMK): Moving one level below, the DMK resides in the master database. It is protected and encrypted by the Service Master Key It is explicitly protected and encrypted by the Service Master Key.
The Certificate or Asymmetric Key: This certificate is created in the master database and is directly protected by the Database Master Key. This certificate is the custodian of the final key.
The Database Encryption Key (DEK): This is the worker bee of the show. The DEK is a symmetric key kept in the user database itself, and protected by the certificate from the previous stage. The DEK is the real key, and it encrypts and decrypts your raw data pages.

TDE on AWS: RDS vs. EC2

Amazon RDS for SQL Server

When you migrate your SQL Server workloads to AWS, you often have the option of going with a fully managed service, or doing it yourself. Management of TDE gets tremendously simple if you go with Amazon Relational Database Service (RDS).

AWS takes care of infrastructure, patching, and OS layers for you. To enable TDE on RDS, just add the TDE option to an RDS Option Group. It interacts natively with AWS Key Management Service (KMS).

You can select an AWS managed key or generate your own Customer Managed Key (CMK) in KMS. This configuration takes away the operational pain of having to manually manage server certificates, since AWS maintains the rotation and security of the underlying keys.

SQL Server on AWS EC2

To be fair, some teams want complete control of their database environment and run SQL Server directly on Amazon EC2 instances. If you run your databases in EC2 you have to manage the full TDE setup yourself, as you would on-premises.

You will write the T-SQL statements to build the Database Master Key, create the server certificates and initialize the Database Encryption Key. This involves a bit more operational cost but allows you deep customization choices.

For example, you can utilize typical local certificates that are stored inside the instance. You may also configure your EC2 SQL Server to connect with AWS CloudHSM or utilize an Extensible Key Management (EKM) provider to delegate key management to a dedicated hardware security module.

Backup, Restore, and Cross-Account Migrations

The Golden Rule of TDE Restores

One of the most important warnings presented during the meetup session is about backup and restore activities. The backup files (.bak) created from a TDE-enabled database are fully encrypted and has stringent dependencies.

And here's the thing: you can't just take a backup file from a TDE protected database and restore it to a completely new SQL Server instance. If you attempt to, SQL Server will instantly throw an error and interrupt the restoration operation.

Otherwise, for a successful restore of a TDE encrypted database, the destination server must have access to the exact same certificate and private key used to encrypt the Database Encryption Key. If you lose that certificate then your backup files are totally useless.

Cross-Account Restores in AWS

This reliance becomes a big concern when you do cross-account migrations or setup disaster recovery sites on AWS. If you're utilizing Amazon RDS and distributing snapshots/native backups across several AWS accounts you need also think about the KMS key permissions.

The source account must explicitly grant the destination AWS account permissions to use the KMS key to decrypt the database backup. For this to function, you will need to add access to the key for the IAM roles in the destination account in the Key Policy for your Customer Managed Key in the source account. Moral of the story: always validate your cross-account restore pipelines before you depend on them in an emergency.

Key Takeaways

Deploying TDE on AWS demands good knowledge of internals of SQL Server and cloud security architecture. Keep in mind these basic operational practices:

Monitor TempDB Behavior: Once you enable TDE on even a single user database on your SQL Server instance, the shared system TempDB is automatically encrypted. This implies that any temporary data spilled out to disk is protected, but it also means that other unencrypted databases on the same server may be slightly affected by the TempDB encryption.
Plan for Performance Overhead: TDE requires additional CPU cycles because the encryption and decryption happen in real time at the I/O operation level. Fortunately, newer AWS infrastructure and Intel/AMD processors provide hardware accelerated encryption (such AES-NI) and therefore the real CPU overhead is little, in the 3-5% range depending on how read/write intensive your workload is.
Backup Your Keys Relentlessly: If you are running SQL Server on EC2, be sure to back up your Database Master Keys and server certificates as soon as you create them. Store them securely in a service such as AWS Secrets Manager or an encrypted Amazon S3 bucket, outside of the instance.

Conclusion

At the end of the day, protecting your database storage layer shouldn’t be a scary process that slows down development. Transparent Data Encryption is an easy, effective approach to meet tough compliance standards and protect sensitive customer data without forcing your engineering staff to update any code.

Whether you use the automation of Amazon RDS or have fine-grained management on Amazon EC2, understanding the internal hierarchy of keys helps keep your data safe from unauthorized access. If you don’t have encryption at rest enabled for your production databases yet, include it as a work item in your next infrastructure sprint.

About the Author

As an AWS Community Builder, I enjoy sharing the things I've learned through my own experiences and events, and I like to help others on their path. If you found this helpful or have any questions, don't hesitate to get in touch! 🚀

🔗 Connect with me on LinkedIn

References

Event: AWS User Group Chennai Meetup

Topic: TDE Inside Out - Protecting SQL server data at rest on AWS

Date: November 15, 2025

Also Published On

AWS Builder Center

Hashnode

From Build to Telemetry - Bedrock Agents with Strands SDK

N Chandra Prakash Reddy — Sun, 28 Jun 2026 13:26:19 +0000

I got the great opportunity to attend the AWS User Group Chennai Meetup on 15 Nov 2025. There were a lot of great sessions throughout the day, but one speaker totally stole the show in my book. "From Build to Telemetry - Bedrock Agents with Strands SDK" was presented by Jaya Ganesh, an Application Developer at Genesys.

I mean, it’s very easy these days to construct a great AI chatbot on your home PC. But to take that AI, give it tools, make it autonomous, and deploy it safely in a production environment? That’s a whole new animal. That sounds familiar?

That was the task for this session. I took a lot of notes and images, and I want to show you exactly how you can turn fragile AI scripts into solid, production-ready systems with Amazon Bedrock and the Strands SDK.

Laying the Groundwork: Generative AI and Bedrock

The session started with the basics and then moved on to complicated agents. The backbone of Generative AI are Large Language Models (LLMs) and Foundational Models (FMs) that understand and generate text based on human cues.

But running these models yourself is a headache. Or to put another way, you want to focus on building features, not managing infrastructure. And that’s exactly what Amazon Bedrock offers. It's a fully managed, serverless API that lets you access industry leading models from Anthropic (like Claude), Meta, Amazon, and more.

More than access to models, Bedrock delivers enterprise-grade security. Your data is kept private and never used to train the base models. It’s also filled with features like as Prompt Management, Prompt Caching, and built-in Guardrails to help keep AI behaviors under check.

Traditionally, you might invoke a model using a simple API call like this:

import boto3
import json

bedrock_runtime = boto3.client('bedrock-runtime', region_name="us-east-1")

response = bedrock_runtime.converse(
    modelId="anthropic.claude-3-5-haiku-20241022-v1:0",
    messages=[
        {"role": "user", "content": [{"text": "Explain AI agents in one sentence."}]}
    ]
)
print(response['output']['message']['content'][0]['text'])

This works really well for simple questions and answers. But here's the thing: It's all reactive. The AI only works when you tell it to.

The Paradigm Shift: Moving to Agentic AI

Traditional chatbots are amazing, but they do require continual human control. Agentic AI is a game changer.

An Agentic AI is a system that operates autonomously and independently to fulfill pre-determined goals. It takes the initiative rather than waiting to be told what to do. It predicts requirements and makes decisions and acts. Imagine your database is like a library . Traditional AI is like a librarian that only answers when you ask a question . Agentic AI is like a librarian that notices when a book is out of place , categorizes it , and updates the catalog without you ever asking .

The Agentic Loop

These agents operate on a continuous, four-step loop:

Perceive: Collects real-time data from APIs or connected systems.
Reason: Uses the LLM to interpret context and plan multi-step actions.
Act: Executes tasks through tool integrations.
Learn: Improves through feedback loops.

Enter Strands SDK and AgentCore

You might be asking yourself how to actually do this without creating thousands of lines of boilerplate code. The answer is in Strands SDK and Amazon Bedrock AgentCore.

AgentCore delivers production ready serverless runtime, controlled state and observability. Strands SDK is the Python framework you use to build on it. Strands offers model flexibility, state management, and simple interface with OpenTelemetry for monitoring.

Here is how simple it is to create your first agent using Strands:

from strands import Agent
from strands.models import BedrockModel

# Create a Bedrock model configuration
model = BedrockModel(
    model_id="anthropic.claude-3-5-sonnet-20241022-v2:0",
    max_tokens=1024
)

# Create an agent
agent = Agent(
    model=model,
    system_prompt="You are a helpful AI assistant specialized in AWS."
)

response = agent("What are the benefits of serverless computing?")

Giving Your Agent a Brain: State and Memory

Without the ability to remember earlier interactions, an agent cannot do complex tasks. Strands handles state in three different ways:

Request State: Context just for a single interaction.
Conversation History: The back-and-forth chat history.
Agent State: Key-value storage for long-term details (like user preferences) that persists across multiple requests.

In production, you can’t merely rely on your laptop’s memory to perform session persistence. Strands provides a FileSessionManager for local testing, however it is straightforward to replace this with an S3SessionManager to store session data on Amazon S3 for distributed cloud environments.

Optimizing the Context Window

LLMs are limited in how much text they can process at once. If your talk goes on too long, the agent will crash or forget stuff. The session shared excellent conversation management ways to solve this.

A SlidingWindowConversationManager can be used to only keep the past 20 messages. Or, better yet, a SummarizingConversationManagerusing a smaller, cheaper model (such Claude Haiku) that intelligently compresses earlier messages while leaving the most current messages alone.

Equipping Agents with Tools

An agent is only as good as the tools it’s working with. Imagine you're ordering meals on Swiggy, you need the app to talk to the restaurant, the delivery guy and the payment gateway.

Strands allows you to construct regular Python functions and simply fit your agent with them with a simple @tool decorator.

from strands import Agent, tool

@tool
def check_shipping_options(zip_code: str, product_id: str) -> dict:
    """Check available shipping options."""
    # Logic to call your shipping API goes here
    return {"method": "Express", "cost": 15.00}

# Pass the tool to the agent
website_chatbot = Agent(
    model=model,
    tools=[check_shipping_options],
    system_prompt="You are a helpful shopping assistant."
)

When the user asks about shipping, the agent automatically reasons that it needs to run the check_shipping_options function, executes it, and formats the output for the user.

Scaling Up: Multi-Agent Patterns

This is where it gets super powerful. Sometimes you give an agent too much responsibility and he gets lost. Or, better yet, you can split and conquer using Multi-Agent patterns.

Workflow Pattern: A choreographed performance. A Researcher agent provides data to an Analyst agent, which provides data to a Writer agent.
Agents as Tools: The “Orchestrator” agent gets the user prompt, then determines to query a specialist Billing Agent or a Tech Support Agent.
Swarm Pattern: A very collaborative environment where an Architect, a Coder and a Reviewer send a task back and forth till it is great.

The golden rule for multi-agent systems? Always specify clear responsibilities, precise time limits, and avoid agents being locked in an unending dialog loop with each other.

Observability and Guardrails

When your agent is out in the wild, you need to know what it is doing. Strands deeply integrates with OpenTelemetry and systems such as Langfuse.

Telemetry setup provides you with three pillars of observability: Traces (end to end flow of the request). Metrics (quantitative data like token consumption and costs). Logs. You may even establish Custom Spans around your tools to monitor exactly how many milliseconds your payment processing tool took to run.

And to keep the AI safe, Amazon Bedrock Guardrails are like the bumpers at a bowling alley. They can be configured to automatically block harmful content, redact PII (e.g. credit card details) and avoid prompt injection attacks. For example, if a user asks a shopping agent about an approaching political election, the Guardrail immediately jumps in and stops the off-topic remark.

The Final Mile: Production Deployment

Long story short, you can’t deploy a local script, thus it’s pointless. Strands and AgentCore make this shockingly easy. Your agent code can be packaged as either a ZIP file (for basic Python setups) or as a Docker container (for heavy dependencies).

You just need to add an entry point decorator around your main function for deployment to AgentCore Runtime:

from bedrock_agentcore_starter_toolkit import Runtime
import boto3

agentcore_runtime = Runtime()

# Configure the deployment
response = agentcore_runtime.configure(
    entrypoint="ecommerce_agent/main.py",
    auto_create_ecr=True,
    requirements_file="ecommerce_agent/requirements.txt",
    agent_name="ecommerce_agent"
)

# Launch it!
launch_result = agentcore_runtime.launch()

When deployed, your application can safely invoke this agent hosted in the cloud with boto3.

Key Takeaways

Some things to take away from this session:

Reactive to Proactive: We're expanding beyond the standard chatbot. Agentic AI is autonomous, anticipates needs, and acts through a continuous cycle of perception, reasoning, action, and learning.
Memory is Crucial: Agents must have brains to be useful. Short-term and long-term state management is a key area to keep your context windows optimum and your agents intelligent.
Divide and Conquer: Don’t overburden one agent. Enhance the reliability and precision of routing jobs to specialized agents with multi-agent patterns like Swarm or Workflow.
Safety and Visibility: You can’t control what you can’t see. For real world production, you must trace execution timings, manage token costs, and enforce Bedrock Guardrails.

Conclusion

In the end, the difference between writing a toy AI script on your laptop and deploying a durable, production-ready AI agent is huge. But tools like Amazon Bedrock AgentCore and the Strands SDK provide a lovely bridge over that divide. Managed memory, simple tool integration, and deep observability provide us what we need to construct scalable, autonomous systems safely.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS User Group Chennai Meetup

Topic: From Build to Telemetry - Bedrock Agents with Strands SDK

Date: November 15, 2025

Also Published On

AWS Builder Center

Hashnode