DEV Community: Harsha Mathan

I Built a Serverless X (Twitter) Quote Bot with AWS Lambda + S3 + DynamoDB

Harsha Mathan — Thu, 19 Feb 2026 05:07:49 +0000

I wanted to build something simple but real: a bot that posts a quote to my X account every day — reliably — without me logging in and copy-pasting.

This post walks through the exact architecture and setup I used:

S3 for storing quotes (quotes.csv)
DynamoDB for tracking which quote is next
Secrets Manager for storing X API credentials
AWS Lambda for the posting logic
EventBridge (optional) to schedule it daily

✅ Result: a serverless bot that posts one quote per run.

Architecture

Flow:

Lambda is triggered (manual test or daily schedule)
Lambda reads quotes.csv from S3
Lambda checks DynamoDB to get the next quote index
Lambda posts the quote to X via API (OAuth 1.0a)
Lambda updates DynamoDB (next_index = next_index + 1)

Prerequisites

An AWS account (I used ap-south-1 (Mumbai))
An X Developer App with Read and Write permissions enabled
Python 3.11 Lambda runtime
A basic CSV file with quotes

Step 1: Create your quotes file (CSV)

Create a file named quotes.csv with a header named quote (one quote per line):

quote
Start where you are. Use what you have. Do what you can.
Small steps every day add up to big changes.
Your future needs you—your past doesn’t.
Breathe. This moment is enough.
Progress, not perfection.
You don’t have to see the whole staircase—just take the next step.
Be kind to yourself; you’re learning.
Consistency beats intensity when it’s done daily.
One good decision today can change your tomorrow.
You are capable of more than you think.

Step 2: Upload the CSV to S3

Create an S3 bucket (private) and upload quotes.csv.

Example:

Region: ap-south-1 (Mumbai)
Bucket: har-vmat-xbot-ap-south-1
Key: quotes.csv

Steps:

Go to Amazon S3 → Buckets → Create bucket
Choose your region (Mumbai / ap-south-1)
Keep Block all public access enabled
Create the bucket
Open the bucket → Upload → select quotes.csv → Upload

Step 3: Create DynamoDB table for state

We’ll use DynamoDB to track which quote should be posted next, so we don’t need to modify the CSV file.

Create a DynamoDB table:

Table name: x_quote_bot_state
Partition key: pk (String)

Steps:

Go to DynamoDB → Tables → Create table
Enter x_quote_bot_state
Partition key: pk (String)
Create table

Now insert the initial state item:

pk = har_vmat
next_index = 0

Steps:

Open the table → Explore table items
Click Create item
Add attributes:
- pk (String): har_vmat
- next_index (Number): 0
Save

Step 4: Store X credentials in AWS Secrets Manager

Store your X OAuth 1.0a credentials in AWS Secrets Manager.

Create a secret:

Secret name: x-bot-credentials
Secret type: Other type of secret

Add these 4 key/value pairs:

consumer_key
consumer_secret
access_token
access_token_secret

Steps:

Go to AWS Secrets Manager → Store a new secret
Choose Other type of secret
Add 4 rows for the keys above and paste the values from your X Developer app
Next → name it x-bot-credentials → store

⚠️ Do not commit these keys to GitHub or hardcode them in Lambda.

Step 5: Create a Lambda Layer for dependencies

Lambda doesn’t include requests-oauthlib by default, so we create a layer containing:

requests
requests-oauthlib

In AWS CloudShell, run:

mkdir -p x-layer/python
pip3 install -t x-layer/python requests requests-oauthlib
cd x-layer
zip -r x-requests-oauthlib-layer.zip python
ls -lh x-requests-oauthlib-layer.zip

Step 6: Create the Lambda function

Create a Lambda function:

Name: x-quote-bot
Runtime: Python 3.11
Architecture: x86_64

Steps:

Go to AWS Lambda → Functions → Create function
Choose Author from scratch
Function name: x-quote-bot
Runtime: Python 3.11
Architecture: x86_64
Under Permissions, choose Use an existing role and select your role (example: x-quote-bot-lambda-role)
Click Create function

Step 6.1: IAM permissions for the Lambda role

Your Lambda execution role must be able to:

Read the CSV from S3
Read the secret from Secrets Manager
Read/write the DynamoDB state item
Write logs to CloudWatch

At minimum, allow these actions:

s3:GetObject on arn:aws:s3:::har-vmat-xbot-ap-south-1/quotes.csv
secretsmanager:GetSecretValue on the secret x-bot-credentials
dynamodb:GetItem and dynamodb:PutItem on table x_quote_bot_state
CloudWatch logs permissions (usually via AWSLambdaBasicExecutionRole)

Step 6.2: Attach the Lambda Layer

Attach your dependency layer (created earlier) to the function:

Open your function → scroll to Layers
Click Add a layer
Choose Custom layers
Select your layer (example: x-requests-oauthlib) and pick the latest version
Click Add

After attaching, you should see it listed with:

Compatible runtime: python3.11
Compatible architecture: x86_64

Step 6.3: Add environment variables

Go to Configuration → Environment variables → Edit and add:

BUCKET = har-vmat-xbot-ap-south-1
KEY = quotes.csv
SECRET_ID = x-bot-credentials
TABLE = x_quote_bot_state
PK_VALUE = har_vmat

Click Save.

Step 7: Add the Lambda code (CSV from S3 → post to X → update DynamoDB)

Open Code → lambda_function.py, replace the contents with the code below, then click Deploy.

import os, json, csv, io, datetime
import boto3
import requests
from requests_oauthlib import OAuth1

s3 = boto3.client("s3")
ddb = boto3.resource("dynamodb")
secrets = boto3.client("secretsmanager")

BUCKET = os.environ["BUCKET"]
KEY = os.environ["KEY"]
SECRET_ID = os.environ["SECRET_ID"]
TABLE = os.environ["TABLE"]
PK_VALUE = os.environ.get("PK_VALUE", "har_vmat")

X_CREATE_POST_URL = "https://api.x.com/2/tweets"

def get_secret():
    resp = secrets.get_secret_value(SecretId=SECRET_ID)
    return json.loads(resp["SecretString"])

def load_quotes_from_s3():
    obj = s3.get_object(Bucket=BUCKET, Key=KEY)
    data = obj["Body"].read()
    try:
        raw = data.decode("utf-8-sig")
    except UnicodeDecodeError:
        raw = data.decode("cp1252")

    f = io.StringIO(raw)
    reader = csv.DictReader(f)

    quotes = []
    for row in reader:
        q = (row.get("quote") or "").strip()
        if q:
            quotes.append(q)
    return quotes

def get_next_index(table, n_quotes):
    resp = table.get_item(Key={"pk": PK_VALUE})
    idx = int(resp.get("Item", {}).get("next_index", 0))
    return idx % max(n_quotes, 1)

def set_next_index(table, next_idx):
    table.put_item(Item={
        "pk": PK_VALUE,
        "next_index": int(next_idx),
        "updated_at": datetime.datetime.utcnow().isoformat() + "Z"
    })

def post_to_x(creds, text):
    auth = OAuth1(
        creds["consumer_key"],
        creds["consumer_secret"],
        creds["access_token"],
        creds["access_token_secret"],
    )
    payload = {"text": text}
    r = requests.post(X_CREATE_POST_URL, auth=auth, json=payload, timeout=20)
    if r.status_code >= 300:
        raise RuntimeError(f"X post failed: {r.status_code} {r.text}")
    return r.json()

def lambda_handler(event, context):
    creds = get_secret()
    quotes = load_quotes_from_s3()
    if not quotes:
        return {"status": "no_quotes"}

    table = ddb.Table(TABLE)
    idx = get_next_index(table, len(quotes))

    text = " ".join(quotes[idx].split())
    if len(text) > 280:
        text = text[:277] + "…"

    resp = post_to_x(creds, text)
    set_next_index(table, idx + 1)

    return {"status": "posted", "index": idx, "tweet": resp}

Step 8: Test it

Now that your Lambda code, environment variables, IAM role, and layer are all set, it’s time to test an end-to-end run.

Steps:

Open AWS Lambda → Functions → x-quote-bot
Go to the Test tab
Create a new test event:
- Event name: test
- Event JSON: {}
Click Test

Expected results:

Lambda returns something like:

{
  "status": "posted",
  "index": 0,
  "tweet": {
    "data": {
      "text": "Start where you are. Use what you have. Do what you can.",
      "id": "YOUR_TWEET_ID"
    }
  }
}

A new post appears on your X profile
DynamoDB item pk = har_vmat updates:
- next_index increments (e.g., 0 → 1)

Conclusion

That’s it — you now have a working, serverless X quote bot powered by AWS:

S3 stores your content (quotes.csv)
DynamoDB tracks which quote is next
Secrets Manager securely stores your X credentials
Lambda posts to X using OAuth 1.0a
EventBridge runs it on a daily schedule (optional but recommended)

The best part: once it’s set up, it’s boring (in a good way). It just runs.

If you’re building your own version, I’d recommend starting simple (like this), and only then adding improvements like randomization, richer content types, or alerting.

AWS Trends to Watch in 2026: Agentic AI, FinOps, Serverless, and Sustainable Infra

Harsha Mathan — Sun, 25 Jan 2026 16:20:41 +0000

2026 is shaping up to be the year AWS “building blocks” start behaving more like systems: AI that can plan and act, cloud spend that’s governed like product quality, serverless that streams and scales for real-time experiences, and sustainability metrics that finally show up in the same conversations as latency and cost.

This post summarizes notable themes AWS has emphasized recently across official announcements, “What’s New” updates, and AWS blog deep dives (AWS News, Machine Learning, Cloud Financial Management, Compute/Serverless, and Infrastructure & Sustainability).

Let’s break down four trends—and what to do about each.

1) Agentic AI goes from demos to production systems

The hype cycle is real, but the platform pieces are getting more concrete: agents, tool use, memory, identity, governance, and async orchestration.

What’s changing in 2026

Agents become an application tier, not a feature. Instead of “chat with your data,” teams are shipping agents that:

take goals and constraints,
select tools,
execute multi-step workflows,
and report outcomes (with guardrails).

A key shift: treating agent runs like reliable jobs (traceable, retryable, auditable), not like a single chat completion.

What you should build toward

Pattern: “Agent + event loop” (async by default)

For production, treat agent runs like jobs:

kickoff event
plan
tool calls happen in steps
results stream back
failures retry safely

This reduces user-facing latency and makes “long thinking” feasible without timeouts.

Guardrails move left

In 2026, the winning teams will bake in:

least-privilege identity for tool calls
audit trails for decisions and actions
bounded autonomy (what an agent may do)
cost and rate limits per agent/task

Practical checklist

[ ] Make agent runs idempotent (retry-safe)
[ ] Log every tool call with inputs/outputs (redact secrets)
[ ] Add “stop conditions” (budget, time, max steps)
[ ] Default to async orchestration (queues + workflows)
[ ] Separate “planner” from “executor” in your design

2) FinOps becomes continuous spend engineering (and AI joins the loop)

In 2026, cost management stops being a monthly report and becomes:

a daily operational practice,
a product metric,
and something teams can query conversationally—with governance.

What’s changing in 2026

The trend is clear: more mature allocation and governance workflows, and cost insights becoming easier to operationalize. Practically, that means teams are moving from “reduce the bill” to engineering unit economics.

What you should build toward

Treat cost like reliability: SLOs, alerts, and ownership

A simple model that works:

Each workload has a cost owner
Define cost SLOs (e.g., “<$X per 1k requests” or “<$Y per tenant per day”)
Alert on drift, not just spikes

Normalize “unit cost”

Instead of “our bill went up,” push toward:

cost per order
cost per inference
cost per active user
cost per GB processed

Make cost explainable

A surprising amount of waste is “unknown unknowns.” Your goal is not just savings—it’s predictability.

Practical checklist

[ ] Define 2–3 unit cost metrics per product
[ ] Add budget + anomaly alerts per team/workload
[ ] Standardize tagging + allocation rules
[ ] Create a weekly “top changes” cost review (15 minutes)
[ ] Treat optimizations as backlog items with owners and deadlines

3) Serverless grows up: streaming, event enrichment, and AI-native architectures

Serverless isn’t just “Lambda + API Gateway” anymore. In 2026, it’s the default fabric for:

event-driven systems,
real-time experiences,
and AI apps that need to stream results fast.

What’s changing in 2026

The design center is moving toward stream-first UX and event-first backend design:

stream partial results quickly
run long tasks asynchronously
keep services loosely coupled via events

What you should build toward

Pattern: Stream-first user experiences

Users tolerate longer responses if they see progress quickly.

Architect for:

fast time-to-first-byte
progressive rendering
partial results + follow-up refinement

Pattern: Event enrichment

Instead of pushing raw triggers downstream, enrich events early so consumers stay simple and reusable.

Practical checklist

[ ] Prefer streaming for any response > 1–2 seconds
[ ] Use queues/workflows for long tasks; stream progress
[ ] Enrich events early so consumers stay simple
[ ] Design for retries, dedupe, and dead-letter handling
[ ] Measure: TTFB, p95 latency, error rate, cost per request

4) Sustainable infrastructure becomes measurable (and measurable becomes manageable)

Sustainability is shifting from goals to instrumentation.

What’s changing in 2026

The big trend: sustainability metrics are becoming easier to attribute and discuss alongside cost and performance—so teams can make practical tradeoffs, not vague promises.

What you should build toward

Sustainability joins the non-functional requirements list

In 2026, mature teams treat sustainability like:

security
reliability
cost
performance

Meaning: tracked, reviewed, improved continuously.

Pattern: Reduce waste first (it helps cost too)

If you do nothing else:

right-size
remove idle resources
prefer managed services when appropriate
match capacity to demand

Practical checklist

[ ] Review carbon metrics quarterly and align them to workload ownership
[ ] Tag workloads so carbon and cost can be attributed together
[ ] Optimize idle and overprovisioned resources first
[ ] Bake sustainability checks into architecture reviews
[ ] Track improvements like you track cost savings (because they’re related)

The “2026 architecture”: Agents + Serverless + FinOps + Sustainability

Here’s the meta-trend: these four areas are converging.

A practical “north star” architecture looks like this:

Agentic layer decides what to do
Serverless event fabric executes steps reliably and scales instantly
FinOps telemetry measures unit cost and prevents budget surprises
Sustainability metrics provide carbon visibility and reduction targets

The companies that win in 2026 won’t be the ones adopting the most services—they’ll be the ones building the tightest feedback loops.

Quick-start plan

If you want a simple, high-leverage sequence:

1) Pick one workflow and build an async agent prototype (bounded scope, strict permissions).

2) Put it behind serverless orchestration and stream progress/results to users.

3) Define unit cost for the workflow and add budget/anomaly alerts.

4) Treat waste removal (idle, overprovisioned, unnecessary data movement) as roadmap items.

Closing thought

2026 isn’t about picking one trend. It’s about building systems where:

AI can act safely,
serverless can deliver instantly,
cost is governed continuously,
and sustainability is measurable and improvable.

What trend are you most excited (or worried) about in 2026—agents, FinOps, serverless streaming, or sustainability?

My Experience as a Speaker at Amazon, Hyderabad

Harsha Mathan — Thu, 20 Nov 2025 12:00:09 +0000

A few days ago, I had the privilege of speaking at AWS Data Day, hosted at the Amazon HQ in Hyderabad.

This wasn’t just another event for me — it turned into one of the most defining moments of my career.

Being invited as a speaker, and in fact the only non-AWS speaker for the entire day, was both humbling and energizing.

I presented on a topic deeply connected to my work at Verisk:

“Our SQL Server to Snowflake Migration Journey using AWS DMS — Best Practices, Pitfalls, and Lessons Learned.”

🎤 The Preparation

I wanted this session to be more than a technical walkthrough.

So I focused on:

Storytelling
Real-world failures and mistakes
Architecture decisions
Lessons learned the hard way
Practical patterns teams can apply immediately

I spent days refining the narrative, improving diagrams, and making sure the session was valuable and relatable for engineers and architects.

🏟️ The Event Day: A Surge of Energy

Walking into the AWS office and seeing a room full of cloud engineers, architects, and data practitioners was surreal.

Once I started speaking, something clicked.

It didn’t feel like presenting — it felt like teaching, something I naturally enjoy and have done years ago as a freelance trainer.

The audience engagement was incredible:

People took notes
Asked deep, technical questions
Took photos of the slides
Reacted to the stories
Stayed fully connected throughout the talk

Several attendees later told me:

“Yours was one of the standout sessions today.”

“This answered exactly the challenges our teams are facing.”

“Wish this was a longer session.”

Those words meant a lot.

⭐ The Feedback: 4.9/5 Rating

At the end of the day, my session received a 4.9/5 rating from attendees.

This wasn’t just feedback — it was a reminder of what I truly enjoy:

Simplifying complexity
Explaining concepts clearly
Public speaking
Storytelling
Teaching

This event brought that clarity back to the surface.

🔍 Key Takeaways I Shared

Here are some of the core points from my talk:

✔️ Common pitfalls in AWS DMS

✔️ Architecture choices that matter

✔️ Why “lift and shift” doesn’t work for real migrations

✔️ How to redesign for Snowflake instead of porting blindly

✔️ Handling performance gaps early

✔️ Lessons learned from production-scale workloads

✔️ Frameworks to make migrations predictable

These resonated strongly with the audience, especially the practical examples based on real-world challenges.

🙏 Thank You, AWS & the Community

I’m deeply grateful to:

The AWS team for the invitation
The audience for the engagement
Everyone who messaged afterward with kind words
My colleagues who supported the migration journey
The larger Data & AI community that continues to inspire learning

🚀 What’s Next for Me?

This experience encouraged me to:

Speak at more meetups
Build and share more architecture content
Conduct hands-on workshops
Mentor engineers
Strengthen my voice in the Data & AI community

If you’d like me to speak at your event or meetup, feel free to reach out!

🌟 Final Thoughts

AWS Data Day wasn’t just a speaking opportunity — it felt like a career turning point.

It reminded me of what I love doing, what I’m good at, and what I want to do more of.

If you attended the session, thank you for making it special.

If not, I hope to meet you at another event soon!

Building a Proactive AI Travel Agent on AWS Bedrock AgentCore (Final Part )

Harsha Mathan — Tue, 23 Sep 2025 08:56:06 +0000

This is the final version of my 4-part blog series on using the AWS service Bedrock AgentCore. In this post, I’ll share a high-level overview of what AgentCore is, what we’re trying to accomplish, and how we can unlock its full potential.

If you’re seeing this for the first time, I’ve detailed the development journey in the earlier three posts—you can quickly refer to them if you’d like more depth or clarifications:

Reference to Github : https://github.com/hvmathan/bedrock-travel-agent

That said, you can still follow along with just this post—it’s self-contained and focuses on production deployment with real APIs.

Why Bedrock AgentCore Matters

In this series, I’ve been building an AI travel agent. Now, creating a travel agent chatbot isn’t new, there are plenty of ways to do it. The real difference here is how we deploy it.

We start by developing and testing locally in Python, but once the logic is solid, instead of following the traditional cloud setup,ECS/EKS cluster provisioning, load balancer configuration, auto-scaling, IAM role/policy management, CloudWatch setup, and ALB rules—we bypass all of that.

With AgentCore, all of these happen behind the scenes, automatically. The deployment itself took me less than a minute, which is mind-blowing compared to the days of setup we usually spend on infrastructure.

The Framework Question

Apart from Bedrock AgentCore, the next key choice was the framework. Since I’m working with a multi-agent system, I chose Strands Agent.

But wait, what’s a framework, and why does it matter?

Think of building an AI system like constructing a house:

The AWS cloud (AgentCore) is the land and utilities (electricity, water, internet).
The framework is the architecture + scaffolding, it gives you ready-made walls, wiring, and plumbing so you don’t have to reinvent everything.
You can always build with raw bricks and cement (custom implementation), but that means drawing up blueprints, laying pipes, and wiring everything from scratch. It’s possible, but slow.

Strands Agent was like a pre-built house plan already designed for AgentCore land it fit perfectly. No hacks, no duct tape, no “custom adapters.” Just code the agent logic and deploy.

That’s why Strands became my framework of choice, it aligned directly with AWS Bedrock, unlike general-purpose frameworks such as LangChain or CrewAI, which are more powerful in some ways but require lots of integration work for AgentCore.

File Structure Breakdown

Let me walk through the final project structure and what each file accomplishes. These files are available in GitHub link that I've shared above.

AgentCore Production Files:

app.py - Main AgentCore wrapper with CORS support
orchestrator.py - Core coordination logic (unchanged from local)
planning_agent.py - Enhanced with regex parsing for flight requests
travel_tool.py - Updated with RealFlightAPI integration
flight_api.py - Aviationstack API integration with fallback system
agentcore.yaml - AgentCore deployment configuration
requirements.txt - Dependencies for cloud deployment

Local Development/Testing Files:

mock_travel_api.py - Original mock data for development
booking_tools.py - Early booking agent prototype
planning_tools.py - Standalone planning utilities
run_agent.py - Local testing script
my_agent.py - Initial agent prototype
orchestrator copy.py - Backup of original orchestrator

UI and Configuration:

travel_ui.html - React-based professional interface
Dockerfile - Auto-generated by AgentCore
.env - API keys (not in repo)
venv/ - Python virtual environment

Deployment: Success and Challenges

I first developed and tested the application locally, and it worked exactly as I envisioned. For this, I integrated a third-party API that provides 100 free calls per month—more than enough for our demo purposes (covered in detail in Part 2 of this series).

Now that we’ve thoroughly tested the application in our local environment, it’s finally time to promote it to AWS.

This is where the real magic and power of AgentCore shines through. Remember the contrast I mentioned earlier—traditional deployments require hours (sometimes days) of configuring ECS/EKS clusters, IAM roles, load balancers, scaling policies, and monitoring.

With AgentCore, all of that complexity disappears. What normally takes days can now be done in just a few minutes with a single command.

In the screenshots below, you’ll see how effortlessly the application was configured and launched to AWS using AgentCore—no manual setup, no YAML headaches, just a seamless push from local code to a production-ready environment.

Deploying to AWS Bedrock AgentCore was remarkably smooth:

agentcore configure -e app.py --region us-west-2
agentcore launch

Within minutes, I had:

ARM64 container built in AWS CodeBuild
ECR repository created and populated
IAM roles automatically configured
Production endpoint deployed and ready

We can now see our application up and running on AWS.

I further have the ability the test the same from local usine the below command. The deployed agent works perfectly via CLI:

agentcore invoke '{"message": "Find flights from LHR to CDG"}'

UI Development: Building a Professional Interface

So far, we’ve explored one of the most powerful components of Bedrock AgentCore—the Runtime. To take things a step further, I wanted to give the project a more realistic touch with a proper user interface. After all, what’s an AI booking agent without an engaging front end?

I decided to build a React-based interface that showcases both local and cloud capabilities. It’s packaged as a simple HTML file, but it includes all the core functionalities I envisioned for an AI-powered booking experience as you can see in the below screenshot.

Key Features:

Endpoint switching between local development and cloud deployment
Real-time chat interface with typing indicators
Quick action buttons for common requests
Professional gradient design with AWS branding
Mobile-responsive layout

The UI reveals an interesting architectural decision: while the local mode works seamlessly with HTTP calls, the cloud mode requires AWS SDK integration for proper authentication, making direct browser-to-AgentCore communication complex.

I performed a flight search and it gave me the perfect result as below.

How AgentCore Simplified My AWS Deployment Experience

One of the finest use cases of AgentCore I personally found impressive was how effortlessly we deployed a tested version into AWS production.

Instead of juggling YAML files, IAM roles, and cluster configs, AgentCore’s Runtime automated everything in one command.

What AgentCore Runtime Handled Automatically

Container Building → ARM64 container built in AWS CodeBuild (no Docker setup needed)
ECR Repository → Created and configured without lifting a finger
IAM Roles → Both execution and CodeBuild roles provisioned with proper policies
Auto-scaling → Production-ready scaling configuration applied out of the box
Networking → VPC, security groups, and load balancing automatically wired
Monitoring → CloudWatch logging enabled and integrated
Deployment Pipeline → Complete CI/CD from code → container → running service

Traditional AWS Deployment Would Require

Manual ECS/EKS cluster setup
Load balancer configuration
Auto-scaling group creation
IAM policy writing & role management
ECR repository and permission setup
CloudWatch configuration
VPC and security group management
ALB rules & health check setup

⏱️ Time Comparison

With AgentCore → agentcore launch → 5 minutes total
Traditional AWS → ~2–3 days of infrastructure setup

The Efficiency Achievement

You literally go from local Python code → production-ready, auto-scaling AWS infrastructure in one command.

That’s the core value proposition of AgentCore Runtime:

✅ Abstract away infrastructure complexity

✅ Maintain enterprise-grade reliability and security

✅ Run seamlessly on the same AWS backbone powering major applications

And the best part? When your agent responds via:

agentcore invoke

AgentCore Components Assessment

Let me evaluate which AgentCore components I successfully leveraged:

✅ Successfully Implemented

Runtime

Multi-agent system deployed to AgentCore Runtime
Agent status: READY and fully functional
Handles complex orchestration between planning and execution agents

Tools

Real API integration with external services (Aviationstack)
Multi-agent coordination and delegation
Natural language processing for travel requests
Robust fallback mechanisms for reliability

If you’re interested in exploring additional functionalities of Bedrock AgentCore, this project can be further extended with:

What's Next: Enhancement Roadmap

Phase 5 (Future): Complete AgentCore Integration

Memory Service: Implement persistent user preferences and travel history
Identity Service: Add user authentication and personalized experiences
Gateway Service: Route all external API calls through AgentCore Gateway
Enhanced Observability: Custom metrics, dashboards, and alerting

Advanced Features

Proactive Monitoring: Price alerts for saved routes
Multi-modal Input: Voice commands and image processing
Integration Expansion: Hotels, car rentals, weather, and events
Mobile App: Native iOS/Android applications

Lessons Learned

What I'd Do Differently

Start with AgentCore from Day 1: I spent time building local infrastructure that I later had to adapt. Starting with AgentCore patterns would have been more efficient.
Plan Authentication Early: The browser-to-cloud authentication challenge was more complex than anticipated. A backend proxy service would have simplified this.
Implement Memory from the Start: User context and preferences would significantly improve the experience.

What Exceeded Expectations

Deployment Simplicity: The agentcore launch command handling infrastructure setup was remarkably seamless.
Real API Integration: Once properly configured, the Aviationstack integration worked flawlessly with 500 requests/month being generous for development.
System Resilience: The multi-layered fallback system ensures users always get a response, even when external APIs fail.

https://aviationstack.com/login

Final Reflection

This project taught me that AWS Bedrock AgentCore excels at what it promises: rapidly transforming local agent prototypes into production-ready cloud services. The Runtime component is outstanding, handling deployment, scaling, and infrastructure management seamlessly.

The real learning came from understanding the ecosystem complexity beyond basic deployment. Authentication, persistent memory, and advanced observability require thoughtful architecture decisions that go beyond the initial deployment.

My Recommendation

For developers considering AgentCore:

Start here if you want to focus on agent logic rather than infrastructure
Plan authentication patterns early if you need browser-based access
Leverage the ecosystem - AgentCore's value multiplies when using Memory, Identity, and Gateway services together
Begin with CLI access and add web interfaces as a separate layer

The Bottom Line

The journey from local Python scripts to production AWS deployment took just a few weeks, validating AgentCore's promise of accelerated agent development and deployment.

This concludes my 4-part series on building an AI travel agent with AWS Bedrock AgentCore. The complete code and deployment configurations are available in my repository, and I'm excited to see what the community builds with these patterns.

Series Links:

Part 1: Foundation and Architecture
Part 2: Multi-Agent System Design
Part 3: AgentCore Migration
Part 4: Production Deployment and Lessons Learned (This Post)
Aviation stack account creation - https://aviationstack.com/

Thank you,
Harsha

Migrating a Local Multi-Agent Travel System to AWS Bedrock AgentCore(Part 3)

Harsha Mathan — Sun, 07 Sep 2025 17:07:33 +0000

This is the continuation of Blog 1 - https://dev.to/aws-builders/building-a-proactive-ai-travel-agent-on-aws-my-journey-with-bedrock-agentcore-part-1-36c7 and Blog 2- https://dev.to/aws-builders/building-a-proactive-ai-travel-agent-on-aws-my-journey-with-bedrock-agentcore-part-2-199i After successfully building a multi-agent travel planning system locally, I faced the next challenge: integrating it with AWS Bedrock AgentCore for cloud deployment. This post covers the technical journey of wrapping existing Python agents with AgentCore and the lessons learned along the way.

The Challenge: From Local to Cloud-Ready

I had a working travel agent system with multiple components:

Orchestrator: Main coordination logic
Planning Agent: AI-powered trip planning
Booking Tools: Flight and hotel search capabilities
Travel Tools: External API integration

The goal was to integrate this with AWS Bedrock AgentCore without completely rewriting the existing architecture.

Architecture Overview

My local system used a clean multi-agent pattern:

class Orchestrator:
    def __init__(self):
        self.planning_agent = PlanningAgent()
        self.travel_tool = TravelTool()

    def handle_user_request(self, user_input: str) -> str:
        # Coordinate between agents and tools
        plan = self.planning_agent.create_plan(user_input)
        # Execute based on plan...

The AgentCore Integration Journey

Step 1: Installing the Right Packages

The key was identifying the correct AgentCore packages:

pip install bedrock-agentcore
pip install strands-agents  
pip install bedrock-agentcore-starter-toolkit

Step 2: Creating the AgentCore Wrapper

Rather than rewriting my agents, I created a thin wrapper that bridges my existing code with AgentCore:

from bedrock_agentcore import BedrockAgentCoreApp
from orchestrator import Orchestrator

app = BedrockAgentCoreApp()
orchestrator = Orchestrator()

@app.entrypoint
def invoke(payload):
    user_message = payload.get("message", "")
    response = orchestrator.handle_user_request(user_message)
    return {
        "result": response,
        "status": "success",
        "powered_by": "AWS Bedrock AgentCore"
    }

if __name__ == "__main__":
    app.run()

Step 3: Debugging Import Issues

Initially encountered import errors with the Strands framework:

ImportError: cannot import name 'Tool' from 'strands.tools'

The solution was understanding Strands uses lowercase tool decorator, not Tool class:

from strands import tool  # Not from strands.tools import Tool

Step 4: Handling Class Name Mismatches

My wrapper initially looked for TravelOrchestrator but my class was named Orchestrator. The fix was straightforward:

# Instead of:
from orchestrator import TravelOrchestrator
orchestrator = TravelOrchestrator()

# Use:
from orchestrator import Orchestrator
orchestrator = Orchestrator()

Testing the Integration

Once the wrapper was working, testing locally revealed the integration was successful:

python app.py
# Server starts on http://localhost:8080

# Test flight search
curl -X POST http://localhost:8080/invocations \
  -H "Content-Type: application/json" \
  -d '{"message": "Find flights from NYC to Tokyo"}'

Response:

{
  "result": "Flight Search Results:\nSkyWings Flight F101: NYC → Tokyo\nPrice: $349.99",
  "session_id": "default", 
  "status": "success",
  "agent": "orchestrator",
  "powered_by": "AWS Bedrock AgentCore"
}

AWS Configuration and Infrastructure

AgentCore's configuration process is streamlined:

agentcore configure -e app.py

This automatically:

Detected my requirements.txt
Set up IAM execution roles
Configured ECR repository
Generated Docker files for deployment

The configuration summary showed everything was ready:

Region: us-west-2
Auto-created execution role and ECR repository
IAM authorization configured

Deployment Challenges

Attempted cloud deployment with:

agentcore launch

The deployment successfully:

Created ECR repository in us-west-2
Set up IAM execution roles
Configured CodeBuild roles

However, encountered SSL certificate validation issues due to corporate network restrictions:

SSL validation failed for https://bedrock-agentcore-codebuild-sources-*.s3.amazonaws.com/
certificate verify failed: unable to get local issuer certificate

Key Technical Insights

1. Wrapper Pattern Works Well

Rather than rewriting existing agents, the thin AgentCore wrapper approach preserved the original architecture while adding cloud capabilities.

2. Import Debugging Strategy

When facing import errors:

Check exact class/function names in your modules
Verify framework-specific import patterns
Use debugging to see what's actually available in modules

3. AgentCore's Infrastructure Automation

The agentcore configure and agentcore launch commands handle significant infrastructure setup automatically:

IAM role creation with proper policies
ECR repository setup
CodeBuild project configuration
ARM64 container building in the cloud

Architecture Benefits Achieved

The integration maintained the original multi-agent benefits while adding:

Scalability: Ready for AWS cloud deployment
Standardization: Uses AgentCore patterns for consistency
Monitoring: Built-in health checks and logging
Production-Ready: Automatic infrastructure provisioning

Next Steps

With the local AgentCore integration working, the next phase involves:

Adding AgentCore Memory for user preference persistence
Implementing proactive monitoring capabilities
Completing cloud deployment from a non-restricted network
Integrating real-time APIs to replace mock data

Code Repository

The complete implementation is available with:

Original multi-agent system
AgentCore wrapper integration
Configuration files for deployment
Testing commands and examples

Conclusion

Migrating existing Python agents to AWS Bedrock AgentCore doesn't require a complete rewrite. The wrapper pattern allows preserving existing architecture while gaining cloud deployment capabilities. The main challenges were understanding framework-specific imports and navigating network restrictions rather than fundamental architectural issues.

The result is a production-ready travel agent that maintains its original multi-agent design while being deployable on AWS infrastructure with auto-scaling, monitoring, and enterprise-grade security features.

This post is part of my journey building AI agents for the AWS Bedrock AgentCore challenge. Previous posts covered the initial multi-agent architecture and planning system design.

Building a Proactive AI Travel Agent on AWS: My Journey with Bedrock AgentCore (Part 2)

Harsha Mathan — Thu, 04 Sep 2025 07:58:42 +0000

Building a Proactive AI Travel Agent on AWS: My Journey with Bedrock AgentCore (Part 2)

🧠 From One Agent to a Multi-Agent System

In Part 1, I walked through the foundation: getting a single AI agent running on Amazon Bedrock AgentCore. That first step gave me a working runtime, a clean HTTP interface, and confidence that I could build a travel concierge on top of it.

But a real travel concierge needs more than just chat. It needs to:

Understand complex travel requests
Plan multi-step itineraries
Search for flights and hotels
Manage bookings
Be modular enough to extend and scale

That’s exactly what Phase 2 is all about.

🏗️ Phase 2: Local Multi-Agent Travel System

In this phase, I focused on building a fully functional travel orchestration system — still local, but structured to integrate cleanly with Bedrock AgentCore.

💻 GitHub repo: bedrock-travel-agent

🎯 Goals

Modular architecture separating reasoning, planning, and execution
A realistic set of travel tools (flights, hotels, bookings) for testing
Clean interfaces that map easily to Bedrock AgentCore’s tool calling
Keep it Bedrock-ready — no rewriting later

🛠️ Core Components

File	Role
`orchestrator.py`	The “brain” — routes user requests to the right agents and tools
`planning_agent.py`	Interprets natural language, extracts intent, decides actions
`travel_tool.py`	Implements flight and hotel search (ready for real APIs)
`booking_tools.py`	Manages bookings: create, confirm, cancel
`planning_tools.py`	Handles budgets, date parsing, itinerary generation
`mock_travel_api.py`	Provides realistic test data locally
`my_agent.py` / `run_agent.py`	Future Bedrock AgentCore integration entry points

🔄 How It Works

User Input → Orchestrator → Planning Agent → Travel Tools → Response

User: "Plan a 5-day trip to Paris"
↓
Planning Agent: Understands intent, breaks it into sub-tasks
↓
Travel Tools: Fetch flights, hotels, estimate budget
↓
Response: Detailed itinerary and costs

🚦 Sample Interactions

User: “Find flights from NYC to London”

Agent: Returns flights with prices and timings.

User: “I need hotels in Tokyo for next week”

Agent: Returns hotels with ratings, prices, and amenities.

User: “Plan a 5-day trip to Paris”

Agent: Suggests day-by-day itinerary, estimated costs, and booking options.

🧩 Architecture Overview

🏁 Current Status: Ready for Bedrock

At this point, I’ve got:

✅ A multi-agent travel orchestration system

✅ Clean separation of logic

✅ Tools and agents that match Bedrock’s tool-calling paradigm

✅ Mock APIs for safe, realistic testing

But everything is still local. AWS doesn’t know about it — yet.

🚀 What’s Next (Phase 3)

In the upcoming phase, I’ll:

Define tool schemas for Bedrock AgentCore
Register these tools and agents with AgentCore
Deploy the entire system into AWS for scaling, routing, and observability
Test real Bedrock-driven multi-agent orchestration in production-like conditions

Please drop you comments in you have any questions while performing the above.

Thank you,
Harsha

Building a Proactive AI Travel Agent on AWS: My Journey with Bedrock AgentCore (Part 1)

Harsha Mathan — Mon, 01 Sep 2025 17:06:43 +0000

Introduction: The Vision of an Intelligent Travel Concierge

AWS recently introduced Bedrock AgentCore, a powerful new capability packed with exciting features. Before diving into development, I highly recommend watching the official AWS YouTube walkthrough on Bedrock AgentCore to gain a solid understanding of its core concepts and potential.

This is Part 1 of a series documenting my journey: from a foundational prototype to a sophisticated, multi-agent system on AWS.

Amazon Bedrock AgentCore covers the following components:

Runtime
Gateway
Memory
Identity
Tools
Observability

This blog focuses exclusively on the Runtime.

In this installment, I focused on the first goal: get a foundational agent running using Amazon Bedrock AgentCore.

Phase 1: Laying the Foundation

Before booking flights or planning itineraries, I needed:

A secure, scalable runtime to host the AI agent.
A clean abstraction to communicate with a Large Language Model (LLM).
A simple HTTP interface, so future systems can invoke it cleanly.

Enter Amazon Bedrock AgentCore—a serverless agent runtime that handles the scaffolding: scaling, security, endpoints, and more. This frees me to concentrate on the actual logic of the agent, not infrastructure plumbing.

The Tools of the Trade

For this phase, I’m using:

Amazon Bedrock AgentCore – the serverless Agent runtime on AWS.
Strands Agent framework – makes wrapping LLM calls in “agents” easy.
Anthropic Claude – the conversational foundation model accessible via Bedrock.
Python 3.10+ – the language for my agent code.

Getting Started: Step-by-Step

Step 1. Environment Setup

Ensure your AWS account is provisioned for Bedrock and AgentCore in a supported region (e.g., us-east-1, us-west-2).
Run aws configure to set up your credentials.
In a terminal, create a Python virtual environment (optional but recommended)
Install the required packages:

pip install bedrock-agentcore strands-agents bedrock-agentcore-starter-toolkit

Step 2. Write the Agent (my_agent.py)

Create a file named my_agent.py and paste in this code:

from bedrock_agentcore import BedrockAgentCoreApp
from strands import Agent

# Initialize the AgentCore application
app = BedrockAgentCoreApp()

# Initialize the agent to use a specific model
agent = Agent(
    model="anthropic.claude-3-sonnet-20240229-v1:0"  # Use a model ID you have access to
)

@app.entrypoint
def invoke(payload):
    """
    Entrypoint for AgentCore: receives JSON payload, invokes LLM, returns response.
    Expecting payload like: {"prompt": "Your prompt here"}
    """
    user_message = payload.get("prompt", "Hello! How can I help you today?")
    result = agent(user_message)
    return {"result": result.message}

if __name__ == "__main__":
    app.run()  # Starts local server (default: http://127.0.0.1:8080)

I have hardcoded the agent here otherewise by default, the strands.Agent() in your my_agent.py uses a default LLM (Anthropic Claude 4.0) through Amazon Bedrock.

Also, please double check on the below settings before executig the code.

Model access must be granted

You need to go to the AWS Management Console → Amazon Bedrock → Model access, and explicitly enable Claude 4.0 (or whichever model you want).

I was struck at point for a while as Claude 4 was not provisioned to my account and hence I ended up in hardcoding to another version.

The below screenshots shows on how to obtain access to Claude 4.0 model.

Your AWS user/role must have the right permissions
You need to attach the following managed policies (as per quickstart):

AmazonBedrockFullAccess
BedrockAgentCoreFullAccess

Step 3. Run the Agent Locally

Start the agent:

python my_agent.py

You should see:

INFO: Uvicorn running on http://127.0.0.1:8080 (Press CTRL+C to quit)

Step 4. Test the Agent with a Prompt

Invoke-WebRequest -Uri "http://localhost:8080/invocations" `
    -Method POST `
    -ContentType "application/json" `
    -Body '{"prompt": "Plan a 3-day trip to Goa!"}'

You should get a JSON response:


{
  "result": {
    "role": "assistant",
    "content": [
      {
        "text": "Here's a suggested 3-day itinerary for a trip to Goa..."
      }
    ]
  }
}

Thats it! Your AgentCore-backed agent is up and responding to real prompts.

The AgentCore approach wraps your logic inside a server that stays alive, waits for prompts, and returns model outputs—making it production-ready and extensible for multi-agent orchestration.

What’s Next?

So far, we have:

Built a living agent service backed by an LLM.

Wrapped it in a clean, HTTP-accessible interface via AgentCore.

✅ Runtime
Gateway
Memory
Identity
Tools
Observability

Next: Phase 2! I’ll introduce real-world capabilities—connecting the agent to travel APIs via Amazon Bedrock AgentCore Gateway so it can start retrieving flights, hotels, and more. Stay tuned for that in the next post of this series.

Thank you,
Harsha

Tokenization & Data Masking using Lambda - The $700 Million Question

Harsha Mathan — Sun, 29 Jun 2025 09:05:25 +0000

Building a Serverless PII Protection Pipeline: From Equifax's $700M Mistake to a Secure Solution

How a simple tokenization pipeline could have prevented one of history's worst data breaches

https://www.youtube.com/watch?v=vMQwMV62gtU

The $700 Million Question

September 7, 2017. Equifax announces a data breach that would forever change how we think about data security. 147 million people's personal information—names, Social Security numbers, birth dates, addresses—exposed to cybercriminals. The aftermath? A staggering $700 million penalty, irreparable reputational damage, and shattered customer trust. Source : https://archive.epic.org/privacy/data-breach/equifax/

The root cause? Yes, it was a missed Apache Struts patch. But dig deeper, and you'll find the real culprit: lack of layered, proactive data protection.

The "What If" That Started It All

What if Equifax had implemented basic tokenization at their data ingestion points? What if sensitive data was automatically scrambled the moment it entered their systems?

The stolen data would have been useless to attackers.

That "what if" haunted me for years and eventually inspired this project: a lightweight, automated PII protection pipeline that could prevent the next Equifax.

The Solution: A Serverless PII Protection Pipeline

I built an event-driven, serverless pipeline that automatically:

✅ Detects PII fields in uploaded CSV files
✅ Tokenizes sensitive data using reversible encoding
✅ Masks data for safe sharing and analysis
✅ Detokenizes when authorized access is needed

Why Serverless?

💰 Cost-effective: Pay only when processing files
🚀 Scalable: Handles 1 file or 1000 files automatically
🔒 Secure: Built-in AWS security and compliance
⚡ Fast: Near-instant processing for typical datasets

Architecture: Simple Yet Powerful


┌─────────────────┐    ┌───────────────────┐    ┌─────────────────┐
│  Raw CSV Upload │───▶│  PII Detector     │───▶│   Metadata      │
│   (S3 /raw)     │    │     Lambda        │    │  (S3 /metadata) │
└─────────────────┘    └───────────────────┘    └─────────────────┘
                                                           │
                                                           ▼
┌─────────────────┐    ┌───────────────────┐    ┌─────────────────┐
│   Detokenized   │◀───│   Detokenizer     │◀───│   Tokenizer     │
│  (S3 /detok)    │    │     Lambda        │    │     Lambda      │
└─────────────────┘    └───────────────────┘    └─────────────────┘
                                                           │
                                                           ▼
                                                ┌─────────────────┐
                                                │   Tokenized     │
                                                │ (S3 /tokenized) │
                                                └─────────────────┘

The magic happens in 4 steps:

Upload: Drop a CSV into S3's /raw folder
Detect: Lambda automatically identifies PII fields
Tokenize: Sensitive data becomes reversible tokens
Access Control: Only authorized users can detokenize

The Tokenization Logic

I chose Base64 encoding for this MVP—simple, reversible, and perfect for proof-of-concept:

import base64

# Transform "John Doe" into a token
original = "John Doe"
token = base64.b64encode(original.encode()).decode()
print(token)  # Output: "Sm9obiBEb2U="

# Reverse it when needed
decoded = base64.b64decode(token).decode()
print(decoded)  # Output: "John Doe"

Smart Masking for Safe Sharing

The detokenizer includes field-specific masking:

def mask_value(field, value):
    if field.lower() == "name":
        parts = value.split(" ")
        return " ".join([p[0] + "*" * (len(p) - 1) for p in parts])

    elif field.lower() == "email":
        username, domain = value.split("@")
        masked = username[0] + "*" * (len(username) - 2) + username[-1]
        return masked + "@" + domain

    elif field.lower() == "phone":
        return "*" * 6 + value[-4:]

Real Data Transformation Examples

Input: Customer Data

Name,Email,Phone,DOB,TransactionID
John Doe,john@example.com,9876543210,1990-01-01,TXN1001
Jane Smith,jane@gmail.com,9123456789,1991-03-22,TXN1002

Step 1: PII Detection → Metadata

["Name", "Email", "Phone"]

Step 2: Tokenization → Safe Storage

Name,Email,Phone,DOB,TransactionID
TOKEN_1,TOKEN_2,TOKEN_3,1990-01-01,TXN1001
TOKEN_4,TOKEN_5,TOKEN_6,1991-03-22,TXN1002

Step 3: Masked Output → Safe Sharing

Name,Email,Phone,DOB,TransactionID
J*** D**,j**n@example.com,******3210,1990-01-01,TXN1001
J*** S****,j**e@gmail.com,******6789,1991-03-22,TXN1002

Notice how non-PII fields (DOB, TransactionID) remain untouched—preserving data utility while protecting privacy.

The Lambda Functions: Event-Driven Excellence

🔍 PII Detector Lambda

def lambda_handler(event, context):
    # Triggered by S3 upload to /raw folder
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    # Analyze CSV headers and content for PII patterns
    pii_fields = detect_pii_fields(csv_content)

    # Save metadata for tokenizer
    s3.put_object(
        Bucket=bucket,
        Key=f"metadata/{filename}_pii_fields.json",
        Body=json.dumps(pii_fields)
    )

🔐 Tokenizer Lambda

def lambda_handler(event, context):
    # Triggered by metadata upload
    # Read original CSV + PII metadata
    # Replace sensitive values with tokens

    for row in csv_reader:
        for field in pii_fields:
            if field in row and row[field].strip():
                row[field] = f"TOKEN_{token_counter}"
                token_counter += 1

🎭 Detokenizer Lambda (with Masking)

def lambda_handler(event, context):
    # Triggered by tokenized file upload
    # Decode tokens and apply field-specific masking

    for row in csv_reader:
        for field in pii_fields:
            if field in row:
                decoded = base64.b64decode(row[field]).decode()
                row[field] = mask_value(field, decoded)

Security & Compliance by Design

🛡️ Multi-Layered Security

Encryption in Transit: All S3 operations use HTTPS
IAM Policies: Granular access control per folder
Audit Trail: CloudTrail logs every operation
Data Segregation: Clear separation between raw/tokenized/detokenized data

📋 Compliance Ready

GDPR Compliant: Right to be forgotten through detokenization
SOX Friendly: Audit trails and access controls
HIPAA Considerations: De-identification through tokenization

Performance & Cost: The Serverless Advantage

⚡ Performance Metrics

Small files (<1MB): 2-3 seconds end-to-end
Medium files (1-10MB): 5-15 seconds
Concurrent processing: Up to 1000 files simultaneously

💰 Cost Breakdown (Monthly)

Lambda executions: $0.20 per 1M requests
S3 storage: $0.023 per GB
Data transfer: Minimal (internal processing)
Total: <$10/month for typical workloads

Compare that to traditional data protection solutions costing thousands per month!

Lessons Learned: Building in the Real World

✅ What Worked Well

Event-driven architecture eliminated complex orchestration
Folder-based organization provided natural data governance
Base64 tokenization was perfect for MVP validation
Serverless approach kept costs minimal during development

🚨 Production Considerations

Base64 isn't cryptographically secure—upgrade to AES-256 for production
Large files need chunking strategies
Cold starts add 1-2 seconds latency
Error handling needs retry logic and dead letter queues

🔮 What's Next?

[ ] AWS KMS integration for enterprise-grade encryption
[ ] Real-time streaming with Kinesis for live data
[ ] Multi-format support (JSON, XML, Parquet)

Try It Yourself: Getting Started

Prerequisites

# AWS CLI setup
aws configure

# Create your bucket
aws s3 mb s3://your-pii-project-bucket

Quick Deploy

# 1. Clone the repo
git clone [your-repo-url]

# 2. Deploy Lambda functions
./deploy.sh

# 3. Test with sample data
aws s3 cp sample-data/customer_data.csv s3://your-bucket/raw/

# 4. Watch the magic happen!
aws logs tail /aws/lambda/pii-detector --follow

The Bigger Picture: Why This Matters

Beyond Technical Implementation

This isn't just about code—it's about changing how we think about data protection:

Proactive vs. Reactive: Instead of adding security as an afterthought, we bake it into the data pipeline
Privacy by Design: Sensitive data is protected from the moment it enters our systems
Democratic Data Protection: Serverless makes enterprise-grade security accessible to everyone

The Equifax Test

Ask yourself: "If attackers breached my system today, would the stolen data be useless?"

With this pipeline, the answer is yes. Tokenized data without the decryption keys is just gibberish.

Join the Movement

Data breaches aren't slowing down—they're accelerating. But so are our tools to fight them.

This project proves that with modern cloud services, protecting PII doesn't require:

❌ Million-dollar budgets
❌ Dedicated security teams
❌ Complex infrastructure
❌ Months of development

It just requires:

✅ Smart architecture
✅ Automation-first thinking
✅ Security by design
✅ A few lines of Python

What's Your Take?

Have you implemented similar data protection patterns? What challenges did you face? What would you build differently?

Drop your thoughts in the comments—let's make data protection the norm, not the exception.

🔗 Resources

[GitHub Repository]:https://github.com/hvmathan/Tokenization-and-Data-Masking

Thank you!
Harsha

Getting Started with Amazon Q and RAG: Build an AI Assistant Using Just S3 Files!

Harsha Mathan — Wed, 04 Jun 2025 18:29:30 +0000

Are you someone who has always wondered what the buzzword RAG is all about?

If you’ve been following the world of Generative AI, chances are you’ve seen it pop up frequently — especially alongside tools like Amazon Q. But if it still feels like a mystery, you're in the right place.

In this blog, I’ll break down three key ideas:

✅ What is an LLM?

✅ What is Amazon Q and how is it different from ChatGPT?

✅ What exactly is RAG (Retrieval-Augmented Generation)?

And then — we’ll make things fun by building a Cricket Q&A Bot powered by Amazon Q, using just a few PDFs about Virat Kohli and RCB!

Let’s Start With LLMs

You’ve probably used tools like ChatGPT, Grok, or Gemini — marveled at how they can answer almost anything you throw at them.

That’s thanks to something called an LLM — a Large Language Model.

As the name implies "Large" --> These models are trained on massive amounts of public data: books, websites, forums, code, and more.

They use that training to:

Understand your questions
Predict contextually accurate responses
Hold conversations like a human would

So far so good — but here’s the catch...

What If You Want AI to Only Use Your Data?

Let’s say you’re a massive Virat Kohli fan. You’ve collected 5 PDFs — articles, stats, bios — all about him and RCB.

Now imagine you want to create a chatbot that answers questions only from those PDFs. Not the entire internet.

No external opinions. Just facts from those documents.

Can ChatGPT or Grok do that?

❌ Not directly.

But Amazon Q can.

✅ It lets you build an AI assistant that works exclusively on your data — PDFs, CSVs, PPTs, etc. — stored privately and securely in S3.

Enter RAG: Retrieval-Augmented Generation

Here’s where the magic happens.

RAG stands for Retrieval-Augmented Generation — and it’s how tools like Amazon Q answer questions accurately from your documents.

Think of it as a two-step AI process:

Retrieve relevant content from your uploaded documents (like searching inside your PDFs)
Generate a natural, human-like response using that content

Simple Analogy:

RAG is like an open-book exam.

The AI doesn’t memorize everything — it looks up answers in your docs, and then explains them in plain English.

This means:

No outdated info from the web
You stay in full control of your assistant's brain

Cricket Bot Example — Powered by RAG & Amazon Q

I built a quick Cricket Q&A Bot using Amazon Q Developer.

Here’s what I did:

Uploaded 5 short PDFs about Virat Kohli and RCB to S3
Created an Amazon Q application (takes <5 mins)
Asked it questions like:

"How many runs did Kohli score in IPL 2016?"
"Has RCB ever won the IPL?"
"Who coached RCB in 2023?"

Why This Is a Game-Changer
With Amazon Q and RAG, you can build assistants that:

Rely only on private data (great for enterprise use)
Scale as you add new content
Work across industries: HR, finance, insurance, healthcare, sports — you name it!

What You Need to Try This Yourself

✅ An AWS account (Developer Tier is enough)
✅ A few PDFs or files you want to build an assistant for
✅ Amazon Q Developer Console
✅ 20 minutes of curiosity!

Demo

Step 1: Prepare Your Data

Get your source material ready. For this example:

I collected 5 short PDFs about Virat Kohli and RCB
Make sure each document is cleanly structured (stats, articles, bios, etc.)

Step 2: Upload Your Documents to S3

Go to your AWS Console
Open the S3 service
Create a new bucket (e.g., cricket-kohli-bot-data)
Upload your PDFs

Step 3: Create an Amazon Q Application

Go to Amazon Q Developer Console
Click Create Application
Name it something like RCB-Kohli-Assistant
Select Private Data as your knowledge source
Add S3 as your data source, and point it to your bucket

Now it’s time to point your data source. But first — make sure to click on “Create Index” — this is required before you can select any data source (like S3 in our case).

Once the index is created, go ahead and choose S3 as your data source.

You can leave all the settings as default and simply click “Create”.

After setup, your data source will appear in the list — don’t forget to click “Sync” to pull in all your source files (your PDFs from S3).

This will take a while (<1 min for me) but depends on the volume of files you wish to sync.

Now we have all set, lets spin up our bot applicaiton.

Once you enable the “Preview web experience,” it opens a webpage where you can start asking questions — just like you would with ChatGPT.

My questions :

if the info is not available in those PDF's it just returns as No answer found.

And that’s it — you’ve just built your very own customized ChatGPT-style chatbot, tailored exclusively to your data!

Final thoughts :

AI isn’t just about answering everything — it’s about answering the right things based on the right data.

Amazon Q gives you that power.
RAG makes it smart and conversational.
And cricket? Well, that just made it fun. 😄

P.S.: I’m well aware that as of June 4th 2025, RCB finally clinched the title after 18 years! But for the sake of this demo, I’m using sample data that isn’t updated beyond early 2025 — just to illustrate how the concept works in theory.

Thanks for reading, please share your thoughts!

Harsha

AI-Powered Resume Shortlister using Amazon Bedrock

Harsha Mathan — Wed, 14 May 2025 17:10:56 +0000

Recently, I came across a situation to fill a position for my team. The moment we opened the role, I received close to 300 resumes for evaluation. While our HR team did their best to manually sift through them based on job description, experience, skill set, and other factors... I realized:
Why not automate this using Generative AI?

And so, this product was born :)

What This Is About

This project explores how Generative AI (via Amazon Bedrock) can help HR teams:

Automatically screen resumes
Evaluate candidate fitment to job descriptions
Provide actionable insights
All in minutes, not days.

The Problem

Recruiters often face:

Hundreds of resumes per job posting
Time-consuming manual screening
Inconsistent and subjective evaluation

The Solution

With the power of Amazon Bedrock + Python + Streamlit, I built a Resume Shortlister that:

Takes resumes (.docx) as input
Evaluates them against a selected job description
Returns:

Match Score
Reasoning
Missing Skills

Tech Stack

Amazon Bedrock (Titan Text - Nova)
Python
Streamlit for UI
Matplotlib, WordCloud for analytics
Pandas for data handling
Boto3 for AWS integration

Folder Structure

How It Works (Code Snippet on a high level)

def build_prompt(resume_text, jd_text):
    return f"""
Human: Evaluate the following resume against this job description. Score the match out of 100 and explain why.

Job Description:
{jd_text}

Resume:
{resume_text}

Return a JSON object with:
- "score": integer
- "reasoning": list of 2-3 sentences
- "missing": list of skills or experiences missing
Respond only with valid JSON.
Assistant:
"""

resume_shortlister.py

import boto3
import json
from docx import Document

# ---------- Step 1: Load Resume ----------
def extract_resume_text(docx_path):
    doc = Document(docx_path)
    return "\n".join([para.text for para in doc.paragraphs])

# ---------- Step 2: Sample Job Descriptions ----------
job_descriptions = {
    "Project Manager": """
We are looking for an experienced Project Manager to lead cross-functional teams...
(Key skills: Agile, Scrum, Budgeting, JIRA, Communication)
""",
    "Software Developer": """
Join our dev team to build scalable backend services...
(Key skills: Python, Java, APIs, AWS, CI/CD)
"""
}

# ---------- Step 3: Build Prompt ----------
def build_prompt(jd_text, resume_text):
    return f"""
You are an AI assistant helping shortlist job candidates.

Here is the job description:
{jd_text}

Here is the resume:
{resume_text}

Task:
1. Rate this resume from 0 to 100 based on its relevance to the job.
2. List the top 3 reasons why this resume is a good or poor fit.
3. Mention any important missing qualifications.

Respond in JSON format:
{{
  "score": <score>,
  "reasoning": ["..."],
  "missing": ["..."]
}}
"""

# ---------- Step 4: Call Bedrock ----------
def call_bedrock(prompt):
    bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

    body = {
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 500,
        "temperature": 0.5,
        "messages": [
            {"role": "user", "content": prompt}
        ]
    }

    response = bedrock.invoke_model(
        modelId="amazon.titan-text-lite-v1",
        body=json.dumps(body),
        contentType="application/json",
)


    output = json.loads(response['body'].read().decode())
    return output['content'][0]['text']

# ---------- Run ----------
if __name__ == "__main__":
    resume_path = "resumes/Project_Manager_1.docx"
    role = "Project Manager"  # change role

    resume_text = extract_resume_text(resume_path)
    jd = job_descriptions[role]
    prompt = build_prompt(jd, resume_text)

    print("\n📤 Sending prompt to Bedrock...\n")
    result = call_bedrock(prompt)

    print("✅ Response from Bedrock:\n")
    print(result)

streamlit_app.py

import streamlit as st
import os
import boto3
import json
from docx import Document
import pandas as pd
import matplotlib.pyplot as plt
from wordcloud import WordCloud
from collections import Counter
from botocore.exceptions import BotoCoreError, ClientError

# Constants
BEDROCK_MODEL_ID = "amazon.titan-text-lite-v1"
REGION = "us-east-1"

job_descriptions = {
    "Project Manager": "We are looking for an experienced Project Manager to lead cross-functional teams...",
    "Software Developer": "We are looking for a Software Developer with experience in front-end and back-end technologies...",
    "Intern": "We are looking for a motivated Intern with a strong willingness to learn and assist in various tasks...",
    "Team Lead": "Looking for a Team Lead to guide and support engineering teams in agile delivery...",
    "HR Manager": "We need a skilled HR Manager to handle talent acquisition, employee engagement, and compliance...",
    "Sales Expert": "Seeking a sales expert with experience in B2B and client relationship management..."
    # Add more roles as needed
}

# --- Helper Functions ---
def extract_resume_text(docx_file):
    doc = Document(docx_file)
    return "\n".join([p.text for p in doc.paragraphs if p.text.strip() != ""])

def build_prompt(resume_text, jd_text):
    return f"""
Human: Evaluate the following resume against this job description. Score the match out of 100 and explain why.

Job Description:
{jd_text}

Resume:
{resume_text}

Return a JSON object with:
- "score": integer
- "reasoning": list of 2-3 sentences
- "missing": list of skills or experiences missing
Respond only with valid JSON.
Assistant:
"""

def get_bedrock_response(prompt):
    client = boto3.client("bedrock-runtime", region_name=REGION)
    body = {
        "messages": [
            {"role": "user", "content": prompt}
        ],
        "anthropic_version": "bedrock-2023-05-31",
        "max_tokens": 1024,
        "temperature": 0.3
    }
    try:
        response = client.invoke_model(
            modelId=BEDROCK_MODEL_ID,
            contentType="application/json",
            accept="application/json",
            body=json.dumps(body)
        )
        response_body = json.loads(response['body'].read())
        return response_body['content'][0]['text']
    except (BotoCoreError, ClientError) as e:
        st.error(f"Error from Bedrock: {e}")
        return None

# --- Streamlit UI ---
st.title("📄 AI Resume Shortlister & Analytics Dashboard")

role = st.selectbox("Select Job Role", list(job_descriptions.keys()))
uploaded_files = st.file_uploader("Upload Resumes", type=["docx"], accept_multiple_files=True)

results = []

if uploaded_files:
    for uploaded_file in uploaded_files:
        st.write(f"📑 Evaluating: `{uploaded_file.name}`")
        resume_text = extract_resume_text(uploaded_file)
        job_description = job_descriptions.get(role, "")
        prompt = build_prompt(resume_text, job_description)

        with st.spinner("Sending to Amazon Bedrock..."):
            raw_output = get_bedrock_response(prompt)

        if raw_output:
            try:
                parsed = json.loads(raw_output)
                score = parsed.get("score", 0)
                reasoning = " | ".join(parsed.get("reasoning", []))
                missing = parsed.get("missing", [])
                results.append({
                    "name": uploaded_file.name,
                    "score": score,
                    "reasoning": reasoning,
                    "missing": missing
                })

                st.success(f"✅ Score: {score}")
                st.write(f"💡 Reasoning: {reasoning}")
                if missing:
                    st.warning(f"❌ Missing Skills: {' | '.join(missing)}")
            except json.JSONDecodeError:
                st.error("❌ Failed to parse Bedrock's response.")
        else:
            st.error("⚠️ No response from Bedrock.")

# --- Analytics Dashboard ---
if results:
    st.subheader("📊 Resume Analytics")

    df = pd.DataFrame(results)

    # Score Distribution
    st.markdown("### 📈 Score Distribution")
    fig, ax = plt.subplots()
    ax.hist(df["score"], bins=10, color="skyblue", edgecolor="black")
    ax.set_xlabel("Match Score")
    ax.set_ylabel("Number of Candidates")
    st.pyplot(fig)

    # Top Candidates
    st.markdown("### 🏆 Top 5 Candidates")
    top_candidates = df.sort_values(by="score", ascending=False).head(5)
    st.dataframe(top_candidates[["name", "score", "reasoning"]])

    # WordCloud of Missing Skills
    st.markdown("### ☁️ Missing Skills WordCloud")
    all_missing = [skill for sublist in df["missing"] for skill in sublist]
    if all_missing:
        wordcloud = WordCloud(width=800, height=400, background_color='white').generate(" ".join(all_missing))
        fig_wc, ax_wc = plt.subplots()
        ax_wc.imshow(wordcloud, interpolation='bilinear')
        ax_wc.axis("off")
        st.pyplot(fig_wc)
    else:
        st.info("✅ No missing skills across uploaded resumes!")

Streamlit Demo (UI Preview)

Once the above command is run, it opens up the UI which has the below features :

Upload multiple resumes
Choose Job Role (Project Manager, Developer, Intern, etc.)
See Match Score, Reasoning, and Missing Skills
View charts

Suppose I'm hiring a Software Developer for my team — I simply select the relevant role from the dropdown menu.

Next, I upload the resumes I’ve received. For demonstration purposes, I’ve included a mix of profiles to test whether Bedrock can accurately identify the most relevant candidates.

Bedrock now takes over and begins evaluating each profile.
If you're curious about what exactly we're sending to Bedrock, you can revisit the Python code — The prompt is designed to be clear and structured. Bedrock then uses Amazon Titan (Nova) under the hood to evaluate the resume and return results.

Once the analysis is complete, we get a structured summary for each resume, including:

Match Score (out of 100)

Reasoning behind the score

Missing skills or experience compared to the job description

This gives instant clarity to recruiters on which candidates are worth shortlisting — all without manually opening a single resume.

{ "score": 85, "reasoning": [ "The resume demonstrates relevant experience as a project manager...", "The candidate has 1 year of experience...", "Highlights skills in team coordination..." ], "missing": [ "No mention of JIRA", "Education section lacks detail" ] }

Finally, the system presents the Top 5 candidates — each with a match score and clear reasoning — making it incredibly easy for the HR team to not just identify qualified profiles, but to confidently select the best of the best.

Impact on HR Teams

Saves time and effort
Transparent and explainable results
Consistent, AI-assisted decision making
Shortlist ready-to-interview candidates faster

Final Thoughts

This project showcases how Generative AI combined with Amazon Nova can revolutionize Talent Acquisition and HR Analytics — automating resume screening, improving decision-making, and saving hours of manual effort.

Whether you're an HR professional, a hiring manager, or an AI enthusiast, this is a glimpse into the future of intelligent hiring.

Feel free to try it out and share your feedback!

Thanks,
Harsha

Amazon Nova Powered AI Claims Assistant

Harsha Mathan — Sun, 20 Apr 2025 06:19:56 +0000

Introduction

If you're looking to experiment with Amazon Bedrock or want a simple hands-on project to get started, this toy use case is a great way to explore how large language models can be integrated into real-world workflows.

Traditionally, when you submit an insurance claim—whether it’s for an auto accident or a medical procedure—it’s first reviewed by a human adjuster. The adjuster evaluates the details, validates supporting documents, and then determines whether the claim is genuine before approving or rejecting it.

In this blog post, we explore how Amazon Bedrock, specifically the Amazon Nova model, can be used to play the role of an adjuster. By leveraging Bedrock through a simple AWS Lambda function, we demonstrate how AI can assist in summarizing claims and assessing potential fraud risks.

For this initial prototype, we’re using AWS Lambda and Amazon Bedrock, but in the future, this project could be extended to include services like API Gateway, Step Functions, S3, and more.

Step 1: Get Model Access from Amazon Bedrock

Before you begin, make sure you have access to the Amazon Titan Nova Lite and Nova Pro models:

Go to Amazon Bedrock in the AWS Console.

Under Model Access, request access to the Nova models if not already enabled.

Step 2: Create the Lambda Function

In VS Code, create a new folder for your Lambda project.

Write your Lambda function code (shared below).

In the AWS Console, create a new Lambda function.

After creation, a default IAM role will be associated with the Lambda. Add a policy to this role to give it Amazon Bedrock full access.

{
  "Effect": "Allow",
  "Action": [
    "bedrock:InvokeModel"
  ],
  "Resource": "*"
}

Step 3: Lambda Function Code (lambda_function.py)

import boto3
import json
import os

bedrock = boto3.client("bedrock-runtime")
MODEL_ID = os.getenv("MODEL_ID", "amazon.nova-pro-v1:0")

def lambda_handler(event, context):
    body = json.loads(event['body'])
    note = body.get('note', '')

    prompt = build_prompt(note)

    response = bedrock.invoke_model(
        modelId=MODEL_ID,
        contentType="application/json",
        accept="application/json",
        body=json.dumps({
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {"text": prompt}
                    ]
                }
            ]
        })
    )

    model_output = json.loads(response['body'].read())
    ai_reply = model_output.get('output', {}).get('message', {}).get('content', [{}])[0].get('text', '')

    return {
        "statusCode": 200,
        "body": json.dumps({
            "summary_and_score": ai_reply
        })
    }

def build_prompt(note):
    return f"""
You are an AI assistant for insurance claims adjusters. Read the adjuster's note and:

1. Summarize the claim in 3–5 bullet points.
2. Highlight any fraud risk indicators (if any).
3. Provide a fraud risk score between 0 and 10.

Adjuster's Note:
\"\"\"{note}\"\"\"

Output Format:
Summary:
- ...
Fraud Risk:
- ...
Score: ...
"""

Don’t forget to add an environment variable in the Lambda settings:

MODEL_ID = amazon.nova-pro-v1:0

Step 4: Testing the Lambda

Use the Lambda test feature in the console to simulate a request. Here's a sample payload you can use:

{
  "body": "{\"note\": \"Claimant reports a scratch on their car in a parking lot. No CCTV footage is available. No other witnesses. Wants to proceed with the claim.\"}"
}

Sample Output

{
  "statusCode": 200,
  "body": "{\"summary_and_score\": \"Summary:\\n- Claimant reports a scratch on their car in a parking lot.\\n- No CCTV footage is available to support the claim.\\n- Claimant is unable to produce any evidence of the incident.\\n\\nFraud Risk:\\n- Lack of evidence or witnesses.\\n- No CCTV footage to corroborate the claimant's statement.\\n\\nScore: 7\\n\\nExplanation: The high fraud risk score is due to the absence of supporting evidence and the inability to verify the claim through surveillance or other means.\"}"
}

How is the Fraud Risk Score Interpreted?

The fraud risk score is a value from 0 to 10, where:

Score Range Interpretation
0–3 Low fraud risk — the claim appears legitimate 4–6 Moderate risk — the claim may need further review 7–10 High fraud risk — the claim has red flags or lacks supporting evidence

As shown above, the Nova model provides:

A clear summary of the claim
Detected fraud indicators
A risk score with reasoning

Conclusion

This toy project demonstrates how Amazon Bedrock and the Nova model can act as an intelligent assistant for insurance claims adjusters. By summarizing claim notes and providing fraud risk analysis, it can help organizations quickly triage and prioritize claims.

This is just a starting point. Future enhancements may include:

Real-time REST APIs via API Gateway
Workflow orchestration using Step Functions
Secure data storage on S3
Automated claim status notifications

By leveraging the Amazon Nova model through Amazon Bedrock, we're able to mimic the judgment of a seasoned claims adjuster—automating the summarization of adjuster notes, detecting possible red flags, and assigning a fraud risk score based on context.

This AI-powered workflow brings multiple benefits:

🧠 Smarter decisions: Nova evaluates claims contextually, helping to reduce oversight and human error.

⚖️ Consistent outcomes: The model applies the same logic across all claims, ensuring fairness and reducing bias.

🕒 Time savings: Automating the initial review process frees up human adjusters to focus on complex or high-value claims.

📈 Scalable operations: This approach can be applied to thousands of claims daily without additional headcount.

If you have any suggestions or scope of improvement, feel free to comment or you can reach me on LinkedIn - https://www.linkedin.com/in/hvmathan/

Thanks for reading!

AWS Step Function Limitations and How to Overcome Them

Harsha Mathan — Tue, 08 Apr 2025 04:15:59 +0000

AWS Step Functions is one of the best serverless cloud services for orchestration. It is user-friendly, easy to adopt, and integrates seamlessly with numerous AWS services. However, like any tool, it has its limitations. This article highlights some common challenges and practical workarounds...

1. Payload Size Limitation
When using services like AWS DMS, retrieving associated tasks for a replication instance via DescribeReplicationTasks can result in a payload exceeding the 256 KB limit, leading to errors or incomplete data handling.

Workaround:
Use AWS Lambda to preprocess or transform large datasets before passing them to the Step Function. Alternatively, store large payloads in Amazon S3 or DynamoDB, and pass a reference (e.g., an S3 URL) instead.

2. No Native Retry for Express Workflows
Express Workflows lack the robust retry and error-handling mechanisms available in Standard Workflows, making them less resilient to transient failures.

Workaround:
Combine CloudWatch metrics with an external monitoring or alerting system to detect and handle failures. Additionally, design your application logic to retry failed tasks as needed.

3. Limited Monitoring
While Step Functions offer basic CloudWatch metrics, they lack fine-grained debugging capabilities, such as detailed logs for every state transition.

Workaround:
Enable detailed logging by sending execution history to CloudWatch Logs. This allows you to monitor workflow behavior, debug issues, and analyze execution history in depth.

4. Choice Workflow State
When handling pagination in workflows, you might encounter a situation where the marker value is null. In such cases, it's crucial to evaluate the IsNull condition first, followed by the NotNull condition. The order ensures correct behavior.

{
 "Variable": "$.possiblyNullValue",
 "IsNull": true
}

This blog from AWS specifies all the necessary choice state that we can leverage. https://docs.aws.amazon.com/step-functions/latest/dg/state-choice.html#amazon-states-language-choice-state-rules

AWS Step Functions remains a powerful orchestration tool with extensive integrations and scalability. By understanding its constraints and implementing these workarounds, you can design robust and efficient workflows that meet your application needs. Thanks for your time!

— Harsha