DEV Community: N Chandra Prakash Reddy

Scaling Custom LLMs on EKS: Trainium and Inferentia2-Powered AI Infrastructure

N Chandra Prakash Reddy — Mon, 13 Jul 2026 15:19:24 +0000

I attended AWS Community Day Kochi on 20th December 2025 and it was full of amazing insights. There were a lot of great seminars throughout the day on different AWS services but one session in particular totally caught my eye. The session featured a technical deep dive, “Scaling Custom LLMs on EKS: Trainium and Inferentia2-Powered AI Infrastructure”.

Here’s the rundown of what was covered and how it all works and why it matters if you’re building AI apps today.

The Challenge: When Do You Outgrow Standard APIs?

Let's face it, using off-the-shelf APIs like OpenAI or Anthropic is great for rapid prototyping. You obtain an API key, you write three lines of code, and you have intelligence embedded right into your program. But what if your app goes viral and takes off?

The speaker hit us with a hard truth straight away: API costs increase dramatically with request volume. Think of it as a cab. It is quite useful for short, occasional journeys. However, if you are driving 100 miles each day, then owning your own automobile makes a lot more financial sense.

Session data suggests the important inflection point is at around 10 million requests each day. That’s when third party APIs become too expensive and building a custom platform is a very practical, affordable necessity.

Enter AWS Purpose-Built Chips

This is a huge scalability problem. To overcome this AWS has come up with purpose-made hardware accelerators that are specifically developed and constructed from the ground up for Generative AI workloads.

We all know what a typical GPU is, but they are expensive to rent and hard to get at scale. AWS has created its own silicon to solve this very problem.

AWS Inferentia (Inf1): It offers the lowest cost per inference in the cloud for Deep Learning models, with costs up to 70% cheaper than normal EC2 instances.
AWS Inferentia2 (Inf2): Specifically designed for large language model (LLM) and diffusion model. It offers up to 40% better price performance than equivalent EC2 instances.
AWS Trainium (Trn1): Your powerhouse for model training. This means you can achieve up to 50% savings on training costs compared to equivalent Amazon EC2 instances.

The Architecture: Bringing it All Together on EKS

Great software is essential to make hardware useful. This is where the AWS Generative AI stack and SDK for Neurone come into play. Neurone SDK bridges third-party applications such as PyTorch, Ray and vLLM to the underlying Trainium and Inferentia hardware.

The worldwide plan showed a very resilient arrangement across regions. Users connect to an Application Load Balancer via a Global Accelerator, and the Application Load Balancer sends the traffic to a Karpenter-managed, dynamically provisioned Amazon EKS cluster. The models are directly served from the hardware nodes using an Nvidia Triton Inference Server in the cluster, running vLLM and the Neurone ML SDK.

The Reality of Training and Serving at Scale

Imagine painting a big canvas all by yourself with a teeny little brush. That's about how it feels training a massive LLM on a single GPU.

The speaker states that training a certain large model on a single GPU may take ~120 days and cost $500k+. A Trainium Cluster provides the ability to run numerous processors in simultaneously. That takes overall training time down to just 14 days, and cuts the cost to $150k - an astounding 70% cheaper.

Just as vital is serving. 500ms latency is a sluggish experience for traditional GPUs. Inferentia2 lowers this down to a 100-200ms latency, making the experience feel nearly quick, all while being 70% cheaper.

Hands-On: The EKS Configurations and Code

This is where it gets good. The speaker dived straight into the live demo, presenting the real setups that make this magic happen.

1. The Kubernetes Magic Line

If you want to use a Trainium or Inferentia device in your Kubernetes deployment, just add one magic line in your resource limits: [aws.amazon.com/neuron](https://aws.amazon.com/neuron): "1" .

Here is what the manifest looks like:

# The Magic Line - Requesting Neuron Device
resources:
  requests:
    cpu: "4"
    memory: 16Gi
    aws.amazon.com/neuron: "1"   # <- Request 1 Trainium/Inferentia device
  limits:
    cpu: "8"
    memory: 32Gi
    aws.amazon.com/neuron: "1"

To bridge the gap between Kubernetes and the physical hardware, you deploy a DaemonSet called the neuron-device-plugin. It automatically discovers the hardware and mounts the /dev/neuron* paths:

# DaemonSet discovers and exposes Neuron devices
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: neuron-device-plugin
  namespace: kube-system
spec:
  template:
    spec:
      containers:
        - name: neuron-device-plugin
          image: public.ecr.aws/neuron/neuron-device-plugin:2.19.16.0
          securityContext:
            privileged: true  # Required to access /dev/neuron*
          volumeMounts:
            - name: device-plugin
              mountPath: /var/lib/kubelet/device-plugins
            - name: neuron-dir
              mountPath: /dev  # Access Neuron devices

2. Protecting Your Budget with Taints

Here’s a huge money saving tip. Trainium on-demand nodes cost around $1.34 per hour. You do not want to run simple system pods like CoreDNS on those pricey ML instances!

Using Terraform, you can apply a taint to block non-ML pods:

# Terraform: Node Group with Taint
taint {
  key    = "aws.amazon.com/neuron"
  value  = "true"
  effect = "NO_SCHEDULE" # Block non-ML pods!
}

Then you just add a toleration in your kubernetes deployment so your ml pods can schedule there:

# Kubernetes: Pod must tolerate to run on Neuron node
tolerations:
  - key: "aws.amazon.com/neuron"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"

nodeSelector:
  node-type: trainium          # Target Trainium nodes
  aws.amazon.com/neuron: "true"

3. Training with PyTorch + Neuron

The real training code with Pytorch and Hugging Face is delightfully simple. Notice how loading the model in bfloat16 offers you a 2x memory save right away:

# Training on Trainium
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model
model = AutoModelForCausalLM.from_pretrained(
    "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    torch_dtype=torch.bfloat16,  # BF16 = 2x memory savings
)

# Assume model_path is defined
tokenizer = AutoTokenizer.from_pretrained(model_path)
tokenizer.pad_token = tokenizer.eos_token

# Simple training loop
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-5)
model.train()

for step, batch in enumerate(dataloader):
    outputs = model(
        input_ids=batch["input_ids"],
        attention_mask=batch["attention_mask"],
        labels=batch["labels"],
    )
    loss = outputs.loss
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()

    print(f"Step {step} - Loss: {loss.item():.4f}")

4. Serving the Model with FastAPI

The speaker exhibited a clean FastAPI wrapper providing the trained model on Inferentia2.

# Inference Server on Inferentia2
from fastapi import FastAPI
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

app = FastAPI(title="TinyLlama Inference API")

# Load model once at startup
model = AutoModelForCausalLM.from_pretrained("/models/tinyllama")
tokenizer = AutoTokenizer.from_pretrained("/models/tinyllama")

@app.get("/health")
async def health():
    return {"status": "healthy"}

@app.post("/v1/generate")
async def generate(prompt: str, max_tokens: int = 100):
    inputs = tokenizer(prompt, return_tensors="pt")

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_tokens,
            temperature=0.7,
            do_sample=True,
        )

    return {
        "generated_text": tokenizer.decode(outputs[0], skip_special_tokens=True)
    }

The Bottom Line: Math and Money

Your workload is the single most important factor for your memory needs. For a 7B parameter model running in BF16:

Inference Memory: Needs about 16.8 GB of memory. This fits nicely on an inf2.xlarge instance.
Training Memory: You need a big 56 GB because you need to save gradients and optimisers.

The final pricing comparison was the most jaw-dropping slide. APIs with significant traffic can easily run to $60,000/month off-the-shelf. Running your own EKS cluster with Inf2 On-Demand instances reduces that to $547.

But the true game-changer is that you are able to set up Spot Instances with Terraform.

# Terraform: Enable Spot Instances
resource "aws_eks_node_group" "trainium" {
  instance_types = ["trn1.2xlarge"]

  # THE MONEY SAVER 
  capacity_type = "SPOT"  # vs "ON_DEMAND"

  # Scale to zero when not training
  scaling_config {
    desired_size = 0      # Start at zero!
    min_size     = 0      # Allow scale to zero
    max_size     = 1
  }
}

EKS with Inf2 Spot instances (which can scale down to zero when idle) brings the monthly cost down to a mere $166. That's a crazy inexpensive $0.17 every 1K inferences!

Key Takeaways

The 10M Threshold: Third party LLM APIs stop making economical sense and specialised infrastructure becomes needed after 10 million daily requests.
Massive Cost Slashes: Move to an EKS cluster with Trainium and Inferentia2 and cut 50% of your training costs and 40% of your inference costs.
Budget Guardrails: Kubernetes standards and tolerations prohibit basic system pods from schedule-squatting on your premium $1.34/hr ML nodes.
Performance Without Compromise: Lowering infrastructure expenses doesn’t mean losing user experience. Inferentia2 readily maintains a latency of 100-200ms and a sub-100ms P99 latency even at a large 70B parameter scale.

Conclusion

Building custom Generative AI infrastructure is no longer some black art for the tech giants with endless cash at the end of the day. AWS has democratised the hardware layer with purpose-built silicon, and the software gap is bridged with open-source tools like the Neurone SDK, making it developer-friendly.

Long story short, if you are establishing an AI firm or trying to extend an engineering staff beyond basic API wrappers, it's well worth the effort to break free from third-party locks by understanding this purpose-built AWS stack.

About the Author

As an AWS Community Builder, I enjoy sharing the things I've learned through my own experiences and events, and I like to help others on their path. If you found this helpful or have any questions, don't hesitate to get in touch! 🚀

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: Scaling Custom LLMs on EKS: Trainium and Inferentia2-Powered AI Infrastructure

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

From Lake to LLM: Building AI-Ready Data with Amazon S3 Tables

N Chandra Prakash Reddy — Sun, 05 Jul 2026 16:56:04 +0000

Participating in the AWS Community Day Kochi on December 20, 2025 was an absolutely fantastic experience. There is always a certain type of energy being surrounded by passionate developers, cloud architects and techies. There were so many amazing seminars throughout the day but as someone who is really interested in data architecture, one particular tech session immediately grabbed my attention.

The title of the talk was “From Lake to LLM: Building AI-Ready Data with Amazon S3 Tables”. Let's be honest, getting your data truly ready for Artificial Intelligence is typically a major hassle. We hear all this excitement about Generative AI, but very few people talk about the filthy plumbing it takes to make it work. It was a breath of fresh air of a session since it handled that precise plumbing issue head-on.

Here's my thorough dive into what I learnt, including the session's insights and a few comments of my own to help break down the more complex aspects.

The Messy Reality of Today’s Data Lakes

The speaker opened the discussion by tackling the elephant in the room: the enormous “Enterprise AI Adoption Gap”. Does this sound familiar? Many of us have encountered this first hand while attempting to construct machine learning models for our firms.

The main concern uncovered in the session is that bad data in S3 is a real blocker to AI adoption. Think of your company's database as a giant public library. If books ( your data ) are just scattered randomly on the floor instead of being carefully sorted on labeled shelves , nobody can find what they need . This is exactly what is happening with traditional data lakes with scattered files and profound inconsistencies and absolute chaos.

What is Actually Blocking AI?

You might be thinking, why can’t we just point a Large Language Model (LLM) at our existing data lake and let it sort things out. The talk nicely outlined the technical blockers:

Raw S3 Lacks Structure: Basic, raw S3 storage has no built-in schema and query semantics. It only stores files, it doesn't know what is in them.
Siloed Workloads: Usually Business Intelligence (BI) teams and AI teams work with completely different storage pathways. This implies you’re paying to duplicate data and those two copies eventually get out of sync.
Data and Schema Drift: Over time, your data format will vary (this is termed "drift"). The slides made clear that schema drift often disrupts downstream pipelines. Data drift also leads directly to wildly uneven AI outcomes.
Lack of Versioning: A simple S3 setup means losing transactional assurances and tight versioning, which makes your machine learning models much less reliable.
The RAG Headache: Implementing Retrieval-Augmented Generation (RAG), which is how you allow an AI search your private papers, often puts large, highly advanced components into your system.

The point is, LLMs are not magic. They really need to have clean, consistent, and highly controlled data in order to work correctly. Feed them rubbish and they'll hallucinate poor data.

The Fix: A Unified AI-Ready Data Foundation

Enterprises are desperate for a uniform data foundation to fix this chaos. We require a single platform that can run both typical SQL queries and advanced analytics workloads, instead of duct-taping different services together.

The speaker stressed that this modern base needs to have native support built-in for RAG, vector embeddings and seamless LLM-driven insights. It needs centralized governance, accurate data lineage (understanding where your data comes from), and reproducibility across all different workloads.

Entering the Modern Lakehouse Approach

This is when things become interesting. The session continued with the “Modern Lakehouse Approach”. In particular, they un-veiled the capabilities of Amazon S3 Tables that natively build on top of the open source Apache Iceberg format under the hood.

That means S3 Tables deliver the ACID transactional guarantees right to your data lake. An ACID transaction is like sending money electronically. When you transfer ₹1000 to a friend, the system ensures that the money is deducted from you and added to your buddy's account concurrently. If the internet goes down mid way the entire transaction is cancelled. It never leaves money floating about in cyberspace. S3 Tables provides your data files with the same unassailable reliability.

This service provides queryable table semantics and strong schema and metadata consistency. It creates strong governance at the data layer itself, providing the all-important uniform base for both analytics and AI.

Structuring the Chaos: The Medallion Architecture

So how do you manage this new strong lakehouse? The speaker was very bullish on the “Medallion Approach” in data engineering. Imagine this as filtering drinking water. You begin with a muddy river, then you run the water through coarse filters, then fine filters. At the end, you have pure and safe bottled water.

Bronze Layer (The River): This is where raw ingestion and historical data are landed straight from streaming sources such as Kafka and Kinesis or batch sources such as Apache Spark and regular CSV/JSON/TXT files
Silver Layer (The Filter): This is where the raw data is thoroughly screened, cleansed and enriched. Null values are dropped, formats are standardized.
Gold Layer (The Bottled Water): Finally the data is converted to company level aggregates. This is the clean, high-quality data that executives and AI models are fed.

This layered pipeline is built on a solid base of data quality and governance, and flows directly into streaming analytics, BI reporting, data science/ML environments, and data sharing platforms.

Seeing it in Action: Sales Analytics Architecture

To be fair, abstract architectural patterns can seem a little frightening. But the presenter simplified it down with a very practical use case: Building a Sales Analytics platform with Customer Feedback.

The architecture diagram showed a beautiful, logical flow:

Data flows from a source S3 bucket via automated ingestion jobs into the Bronze S3 table.
Transformation jobs then move it into the Silver S3 table.
A important step between the Silver and Gold layers is the production of text embeddings. This is the part where client text reviews are converted into numbers so the AI can understand them.
The refined data lands in the Gold layer S3 table.

From that Gold layer, the data is spread out. Connects to SageMaker Unified Studio for large ML workloads. It also interfaces with a Conversational Chat Interface that runs on Anthropic's Claude LLM and a Model Context Protocol (MCP) Server.

Claude and S3 Tables: The Ultimate Chat Interface

The live demo of the MCP Server querying S3 Tables with Claude was definitely the highlight of the session. The demonstration typed a request into Claude asking it to explain the revenue pattern for EMEA in Q2 2024 and to summarize customer comments connected to that.

Claude logged easily into the database and looked over the results. It said total Q2 revenue was $840,000, but quickly noticed a huge 33% revenue loss for “Product A (Enterprise).”

Instead of a human data analyst hunting for the reason, Claude cross-referenced the customer input. It brought to light the instability and app crashes at peak hours of Product A, leaving the team frustrated. It also brought up criticism for “Product B (MidMarket)” about a competitive gap where customers may churn and “Product C (SMB)” for unclear billing and invoicing problems.

In brief, this design makes complex, large database tables instantly usable business insight, simply by conversing with it in simple English.

Why This Makes Your Data "AI-Ready"

The end result? Amazon S3 Tables are meant to be AI-ready.

They give dependable transactions that result in consistent data pipelines. Your AI is not learning from incomplete files.
They allow flexible schema evolution, so you can evolve easily to changing machine learning models without damaging downstream systems.
They actively address the schema drift and lack of versioning that has typically restricted ML reliability.

S3 Tables solve these fundamental infrastructure issues, and deliver the clean, consistent and regulated datasets that LLMs demand.

Enterprise Gains (The Payoff)

Ultimately, embracing an AI-Ready Lakehouse is more than a technological flex, it will mean huge, measurable organizational benefits.

This architecture will allow organizations to view:

Significantly faster delivery of AI and analytics solutions to their customers.
Far more reliable and consistent LLM-generated outputs (reduced hallucination!).
Reduced infrastructure complexity and overall cloud costs.
Better compliance with stringent compliance and corporate governance norms.
Scalable performance specifically designed to handle rapidly growing datasets.

Key Takeaways

If you’re skimming, here’s a fast summary of the most important things you need to know from the session:

Raw storage is not enough: Just throwing files into an S3 bucket results in schema drift and inconsistencies. LLMs require clean, regulated datasets to produce reliable outputs without hallucinations.
Transactions matter: S3 Tables (powered by Apache Iceberg) provides ACID transactional guarantees to your data lake, preventing broken pipelines so your models are training on reliable data.
Layer your data: The Medallion Approach (Bronze, Silver, Gold) is the best technique to turn messy, raw intake into high quality, enterprise level aggregates appropriate for AI consumption.
Unify your foundation: You don’t have to buy and operate different storage routes for regular BI reporting and advanced AI workloads. A modern lakehouse can manage SQL, analytics and LLM-driven insights under one roof.

Conclusion

At the end of the day, your generative AI applications are only as good as the data you provide them. The session at AWS Community Day Kochi was a reminder that the conventional, chaotic data lake is no longer good enough for modern needs.

With a modern lakehouse strategy using Amazon S3 Tables, we can finally simplify infrastructure complexity and generate much higher trust in our LLM outputs. It takes a little planning to get the right design but the scalable performance and strong oversight you obtain is totally worth it. In short, it’s time to improve our data foundations from a storage lake to a real AI engine.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: From Lake to LLM: Building AI-Ready Data with Amazon S3 Tables

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

14x Cheaper AI: A Real-World LLM Distillation Case Study on Bedrock

N Chandra Prakash Reddy — Sun, 05 Jul 2026 13:06:31 +0000

On 20 December 2025, I was fortunate to be a part of the AWS Community Day Kochi. There were fantastic sessions going on throughout the event, but there was one presentation in particular that caught my whole attention and wouldn’t let go.

The speaker came on stage and delivered a big result right out of the gate – they reduced their AI operational costs by 14x using AWS Bedrock. But this wasn’t a highlight reel of quick success. Let’s face it, tech talks that only display the wonderful stuff don’t educate us much. Instead, there was a clear story of how the squad failed again and again on the route to the big victory.

This session was pure gold if you are a developer, or a firm seeking to scale AI features without burning through your entire runway. Here’s a look at the trip, the technological challenges, and how they finally cracked the puzzle.

The Business Problem: A 3-Body Challenge

But before we get to the solution, we need to understand the nightmare the team was dealing with. They named it the “3-Body Challenge.”

The trouble is...they were drowning in data. Specifically, they were being overwhelmed with unstructured communications about cargo bookings.

The emails were bilingual and consisted of a crazy mix of Japanese and English content.
They needed their system to be able to correctly extract 23 complicated entities from these emails, such as Air Waybill (AWB) numbers, Flight Numbers, Weights and Dimensions.

The Accuracy vs. Cost Dilemma

The aim was to carry out real-time automatic Named Entity Recognition (NER). The system needed to be low latency, and have a very high accuracy rate of over 95% (f1 score), to be useful in the production pipeline.

You might be wondering why not just throw a Large Language Model (LLM) at it? They did well. And the LLM readily met the precision required. But the operating cost at that high volume was a deal breaker.

Sound familiar? This is a trap many teams get into. They had designed a system that worked well, but they knew they could never build a business around a 14x cost problem.

Rethinking the Core Problem

The first important change in their thinking was in how they approached Named Entity Recognition. Instead than using typical BIO (Beginning, Inside, Outside) tagging, they defined NER as a Sequence-to-Sequence (Seq2Seq) task.

Generating Structured JSON

To grasp this, picture it like an e-commerce checkout system. You don’t want the system to just highlight random goods in a shopping cart, you want it to create a well-formed receipt.

The input (sequence 1) in their case was the raw, jumbled email text prompt asking the model to extract all 23 entities as a JSON array. The expected output (sequence 2) was the exact JSON text produced that matched those entities.

The Technical Goal: Knowledge Distillation

To achieve the high accuracy of a big LLM without the massive price, they turned to a concept known as Knowledge Distillation.

Think of your database as a huge library and the “Teacher Model” as the chief librarian who has read and comprehended every book. The teacher is large, complex and expensive to consult. The purpose of distillation is to compress the knowledge and transfer it to a “Student Model”. The student is smaller, considerably faster and much cheaper to run, offering you the best of both worlds, great precision and low cost.

Evaluating the Distillation Options

The speaker outlined the major routes he may take to achieve this knowledge transfer:

Option 1: Logit-Based (e.g., DistilBERT): This method uses a metric called KL Divergence to match the student's final output probabilities (logits) to the teacher's. It is easy, fast and effective. But it typically misses a lot of the sophisticated internal “reasoning” of the teacher model.
Option 2: Feature-Based (e.g., TinyBERT): That is, to align the internal hidden states and attention mappings of the two models. these transfer knowledge really deep. The negatives? It's quite brittle. It requires model architectures to be same and is quite sensitive to throwing shape_mismatch errors.
Option 3: Token-Based: Here, the teacher model is compared with the final output sequence of the pupil token by token. It learns from the teacher's soft labels and is suitable for generative Seq2Seq jobs such as the JSON extraction they needed.

They decided to go with the Token-Based method as they were generating JSON arrays. And now it gets interesting, and by fascinating I mean extremely frustrating for their engineering team.

The Engineering Nightmares

Attempt 1: The Token Mismatch Wall

Their initial approach was a custom token distillation built with PyTorch. They sought to distill the Llama 3-8B model to the Llama 3-1B model with their specialized Seq2Seq task.

In fairness, the logic was reasonable, but the technological reality was a failure. For token-based distillation, you need absolute, perfect token alignment in order to effectively distill that output JSON. They ran into a big problem: the Llama 3 tokenizer and their Japanese/English bilingual text were misaligned.

They were trying to compare output sequences that just didn’t line up, and the training loss got wildly unstable. This caused a constant token_mismatch_error.

Attempt 2: The Brittle Architecture Wall

Try 2. No way they were giving up. They tried to run a Logit/Feature-Based distillation technique with a library named TextBrewer.

This, too, died a technical death, as the solution was too brittle and required completely matching architectures. The library was quite strict about requirements and their specific Llama models were incompatible.

The operation failed again, generating a shape_mismatch_error. The team found that they were spending 100% of their time fighting engineering difficulties and 0% of their time on true data research.

The Pivot: Isolating the Real Problem

The team stepped back and saw their problem was not a terrible theory. Their problem was bad engineering. They were getting beaten on the two hardest segments of the pipeline:

Tokenizer and Architecture Alignment.
Setting up stable, distributed training environments.

In short, they required a completely managed solution to do the heavy lifting on the engineering requirements things like framework selection, dataset preparation, and normalizations so they could focus entirely on addressing their business challenge.

Enter Amazon Bedrock Model Distillation

They migrated their whole pipeline to Amazon Bedrock. The new method looked like this:

Take the user prompts and feed them into the large Teacher model.
Generate high-quality synthetic data.
Use that data to train the smaller Student model and transfer the knowledge.
Deploy the customized distilled model for real-world inference.

The workflow was pleasantly simple. The prepared prompt dataset was just taken by a data scientist and turned into a JSONL file and uploaded to an Amazon S3 bucket. They then picked whatever instructor and student models they wanted from within the Amazon Bedrock service to start the distillation job.

What made this one work while all the others didn't? It wasn’t magic, the speaker pointed out. I checked the CloudWatch training logs and it just... worked.

It solved alignment: By using the compatible Nova Pro and Nova Lite model family, there was zero token mismatch.
It solved stability: The managed service handled all the complex orchestration behind the scenes.

Their training loss went from 0.05 to a very accurate 0.008 in just 4 epochs and 70 total steps.

The Results: Fast, Accurate, and Cheap

They ran the newly distilled Nova Lite model that took the jumbled input prompt and produced a perfectly formatted JSON output array of the desired entities.

The bottom line? The stats tell the story:

The Teacher (Nova): Achieved a 97% overall F1 score (96.3% English, 95.4% Japanese) but cost 14x more to run.
The Student (Nova Lite): Achieved a 95.085% overall F1 score (96.535% English, 93.635% Japanese) at the 1x baseline cost.

They hit their >95% accuracy goal while entirely eliminating the 14x operational cost overhead!.

A Deeper Look at the Errors

Always seeking improvement, the team took a further look at the little 1.9% accuracy disparity between teacher and student.

Language: The student did significantly worse on the Japanese text (93.6% vs 95.4% for the teacher). Their next immediate step to fill the gap is to extend and increase the modest 150-sample Japanese dataset.
Complexity: Approximately 20% of the student’s errors were in the “long-tail” or multi-part entities. For example elaborate layered instructions such as “Fragile; refrigerate below 4°C” In these tough edge instances, the teacher model’s richer baseline reasoning nevertheless came out on top.

Key Takeaways

Generative NER is a game-changer: By moving from typical BIO tagging to a Seq2Seq technique, you gain amazing flexibility when you need to handle complex multi-entity extractions.
Your task dictates your method: If you pick a Seq2Seq task you are forced in a Token-Based distillation technique. Just be ready for the brittle engineering and alignment needs that are part of it.
The real win is cost, not speed: The big 14x improvement was about operational savings. Both models had similar inference performance but the smaller Nova Lite reduced the hefty financial burden of provisioning large LLM throughput.
Offload the engineering: If your team spends 100% of its time resolving architecture incompatibilities, they’re not conducting data science. Get rid of those inflexible tokenizer bottlenecks altogether with managed services like AWS Bedrock.

Conclusion

Sitting in the audience in Kochi at the conclusion of the day, I was reminded that the road to a massive technical win is seldom a straight line. The team didn’t just build a 14x cheaper AI, they failed forward, knew when they were in an engineering trap, and pivoted to a managed solution that allowed them to focus on solving their underlying business challenge.

If you’re producing AI solutions, don’t be scared to change your architecture when operational costs start to threaten your business model. At times, the smartest technological option you can make is simply to let a managed service do the heavy lifting.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Community Day Kochi

Topic: 14x Cheaper AI: A Real-World LLM Distillation Case Study on Bedrock

Date: December 20, 2025

Also Published On

AWS Builder Center

Hashnode

TDE Inside Out - Protecting SQL server data at rest on AWS

N Chandra Prakash Reddy — Sun, 28 Jun 2026 15:44:20 +0000

It was an awesome day at AWS User Group Chennai Meetup on 15th November, 2025. There is always something special about getting together with the local community, sharing real-world stories, and geeking out over cloud architecture. There were several interesting lectures throughout the day on a wide variety of topics including serverless and containers, but one particular talk on database security particularly got my attention.

Database security can seem complex, especially with enterprise workloads, but this webinar did a great job of breaking it down. Let’s lay down exactly how you can protect your Microsoft SQL Server data at rest on AWS without breaking a sweat.

The Core Challenge of Data at Rest

Let’s face it… we spend a lot of effort on network perimeter security. We set up firewalls, maintain security groups, and ensure that all data in transit is encrypted using TLS. But what if someone gets hold of your underlying storage backup or a raw database snapshot?

Do you know this? Security is frequently an afterthought until there is an audit or breach. When your raw database files are stored on the storage layer unencrypted, anyone with access to those files can read your sensitive data. This is where the concept of securing data “at rest” is non-negotiable.

Imagine your database is a bank with strict security. Locking the main door and checking IDs at the entrance, is like protecting data in transit. But if the cash is just lying around on open tables inside the vault, then the whole world wins when they sneak past the front door. Protecting data at rest is like putting that cash in separate, reinforced lockboxes inside the vault.

What Exactly is Transparent Data Encryption (TDE)?

The Basics of I/O Level Protection

That’s where Transparent Data Encryption, or TDE, comes in. TDE is a capability that is incorporated into Microsoft SQL Server. It encrypts the real files that are written to disk to protect your data. The beauty of TDE is in the name, it is totally transparent to your application.

You might be thinking, what does “transparent” look like in practice. In other words, you do not need to rewrite your application code, update your SQL queries or change your database schemas. SQL Server does all the hard work for you, behind the scenes.

SQL Server receives data from disk, decrypts it on the fly, and puts it into memory when your application requires data. As data is refreshed back to the storage layer, SQL Server encrypts the data as it hits the disk. This procedure is completely I/O based . It is intended for page level encryption .

What Gets Protected?

When you turn on TDE on a database it’s not just the major tables that are encrypted. This safeguards the whole database ecology on that instance. TDE actively protects the following key elements from session information:

Data Files (.mdf): The main storage files where your actual tables, indexes and user data reside.
Log Files (.ldf): The transaction logs that record all the changes made to the database, and are critical for recovery.
Backup Files (.bak): Any native backups of databases that you produce are automatically encrypted at the time of creation.
TempDB: The system database for temporary items, internal joins and sorting.

Unpacking the Encryption Hierarchy

The Chain of Trust

Here’s where things gets interesting. TDE does not depend on a single password or a single key for the security of your data. Instead, it uses a complex, multi-tiered encryption system sometimes referred to as a chain of trust.

This hierarchy is like a classic Russian nesting doll. You have to unlock a bunch of bigger dolls to get to the tiny doll inside (your real data). If one layer is broken or missing, the entire chain breaks, maintaining the security of your data.

Breaking Down the Layers

So let's walk through how these keys work together from the top down to develop a strong security model:

The Service Master Key (SMK): This is the root of the entire encryption hierarchy . It is automatically created at the very top level when the SQL Server instance is first setup, and is encrypted by the underlying Windows operating system or AWS infrastructure.
The Database Master Key (DMK): Moving one level below, the DMK resides in the master database. It is protected and encrypted by the Service Master Key It is explicitly protected and encrypted by the Service Master Key.
The Certificate or Asymmetric Key: This certificate is created in the master database and is directly protected by the Database Master Key. This certificate is the custodian of the final key.
The Database Encryption Key (DEK): This is the worker bee of the show. The DEK is a symmetric key kept in the user database itself, and protected by the certificate from the previous stage. The DEK is the real key, and it encrypts and decrypts your raw data pages.

TDE on AWS: RDS vs. EC2

Amazon RDS for SQL Server

When you migrate your SQL Server workloads to AWS, you often have the option of going with a fully managed service, or doing it yourself. Management of TDE gets tremendously simple if you go with Amazon Relational Database Service (RDS).

AWS takes care of infrastructure, patching, and OS layers for you. To enable TDE on RDS, just add the TDE option to an RDS Option Group. It interacts natively with AWS Key Management Service (KMS).

You can select an AWS managed key or generate your own Customer Managed Key (CMK) in KMS. This configuration takes away the operational pain of having to manually manage server certificates, since AWS maintains the rotation and security of the underlying keys.

SQL Server on AWS EC2

To be fair, some teams want complete control of their database environment and run SQL Server directly on Amazon EC2 instances. If you run your databases in EC2 you have to manage the full TDE setup yourself, as you would on-premises.

You will write the T-SQL statements to build the Database Master Key, create the server certificates and initialize the Database Encryption Key. This involves a bit more operational cost but allows you deep customization choices.

For example, you can utilize typical local certificates that are stored inside the instance. You may also configure your EC2 SQL Server to connect with AWS CloudHSM or utilize an Extensible Key Management (EKM) provider to delegate key management to a dedicated hardware security module.

Backup, Restore, and Cross-Account Migrations

The Golden Rule of TDE Restores

One of the most important warnings presented during the meetup session is about backup and restore activities. The backup files (.bak) created from a TDE-enabled database are fully encrypted and has stringent dependencies.

And here's the thing: you can't just take a backup file from a TDE protected database and restore it to a completely new SQL Server instance. If you attempt to, SQL Server will instantly throw an error and interrupt the restoration operation.

Otherwise, for a successful restore of a TDE encrypted database, the destination server must have access to the exact same certificate and private key used to encrypt the Database Encryption Key. If you lose that certificate then your backup files are totally useless.

Cross-Account Restores in AWS

This reliance becomes a big concern when you do cross-account migrations or setup disaster recovery sites on AWS. If you're utilizing Amazon RDS and distributing snapshots/native backups across several AWS accounts you need also think about the KMS key permissions.

The source account must explicitly grant the destination AWS account permissions to use the KMS key to decrypt the database backup. For this to function, you will need to add access to the key for the IAM roles in the destination account in the Key Policy for your Customer Managed Key in the source account. Moral of the story: always validate your cross-account restore pipelines before you depend on them in an emergency.

Key Takeaways

Deploying TDE on AWS demands good knowledge of internals of SQL Server and cloud security architecture. Keep in mind these basic operational practices:

Monitor TempDB Behavior: Once you enable TDE on even a single user database on your SQL Server instance, the shared system TempDB is automatically encrypted. This implies that any temporary data spilled out to disk is protected, but it also means that other unencrypted databases on the same server may be slightly affected by the TempDB encryption.
Plan for Performance Overhead: TDE requires additional CPU cycles because the encryption and decryption happen in real time at the I/O operation level. Fortunately, newer AWS infrastructure and Intel/AMD processors provide hardware accelerated encryption (such AES-NI) and therefore the real CPU overhead is little, in the 3-5% range depending on how read/write intensive your workload is.
Backup Your Keys Relentlessly: If you are running SQL Server on EC2, be sure to back up your Database Master Keys and server certificates as soon as you create them. Store them securely in a service such as AWS Secrets Manager or an encrypted Amazon S3 bucket, outside of the instance.

Conclusion

At the end of the day, protecting your database storage layer shouldn’t be a scary process that slows down development. Transparent Data Encryption is an easy, effective approach to meet tough compliance standards and protect sensitive customer data without forcing your engineering staff to update any code.

Whether you use the automation of Amazon RDS or have fine-grained management on Amazon EC2, understanding the internal hierarchy of keys helps keep your data safe from unauthorized access. If you don’t have encryption at rest enabled for your production databases yet, include it as a work item in your next infrastructure sprint.

About the Author

As an AWS Community Builder, I enjoy sharing the things I've learned through my own experiences and events, and I like to help others on their path. If you found this helpful or have any questions, don't hesitate to get in touch! 🚀

🔗 Connect with me on LinkedIn

References

Event: AWS User Group Chennai Meetup

Topic: TDE Inside Out - Protecting SQL server data at rest on AWS

Date: November 15, 2025

Also Published On

AWS Builder Center

Hashnode

From Build to Telemetry - Bedrock Agents with Strands SDK

N Chandra Prakash Reddy — Sun, 28 Jun 2026 13:26:19 +0000

I got the great opportunity to attend the AWS User Group Chennai Meetup on 15 Nov 2025. There were a lot of great sessions throughout the day, but one speaker totally stole the show in my book. "From Build to Telemetry - Bedrock Agents with Strands SDK" was presented by Jaya Ganesh, an Application Developer at Genesys.

I mean, it’s very easy these days to construct a great AI chatbot on your home PC. But to take that AI, give it tools, make it autonomous, and deploy it safely in a production environment? That’s a whole new animal. That sounds familiar?

That was the task for this session. I took a lot of notes and images, and I want to show you exactly how you can turn fragile AI scripts into solid, production-ready systems with Amazon Bedrock and the Strands SDK.

Laying the Groundwork: Generative AI and Bedrock

The session started with the basics and then moved on to complicated agents. The backbone of Generative AI are Large Language Models (LLMs) and Foundational Models (FMs) that understand and generate text based on human cues.

But running these models yourself is a headache. Or to put another way, you want to focus on building features, not managing infrastructure. And that’s exactly what Amazon Bedrock offers. It's a fully managed, serverless API that lets you access industry leading models from Anthropic (like Claude), Meta, Amazon, and more.

More than access to models, Bedrock delivers enterprise-grade security. Your data is kept private and never used to train the base models. It’s also filled with features like as Prompt Management, Prompt Caching, and built-in Guardrails to help keep AI behaviors under check.

Traditionally, you might invoke a model using a simple API call like this:

import boto3
import json

bedrock_runtime = boto3.client('bedrock-runtime', region_name="us-east-1")

response = bedrock_runtime.converse(
    modelId="anthropic.claude-3-5-haiku-20241022-v1:0",
    messages=[
        {"role": "user", "content": [{"text": "Explain AI agents in one sentence."}]}
    ]
)
print(response['output']['message']['content'][0]['text'])

This works really well for simple questions and answers. But here's the thing: It's all reactive. The AI only works when you tell it to.

The Paradigm Shift: Moving to Agentic AI

Traditional chatbots are amazing, but they do require continual human control. Agentic AI is a game changer.

An Agentic AI is a system that operates autonomously and independently to fulfill pre-determined goals. It takes the initiative rather than waiting to be told what to do. It predicts requirements and makes decisions and acts. Imagine your database is like a library . Traditional AI is like a librarian that only answers when you ask a question . Agentic AI is like a librarian that notices when a book is out of place , categorizes it , and updates the catalog without you ever asking .

The Agentic Loop

These agents operate on a continuous, four-step loop:

Perceive: Collects real-time data from APIs or connected systems.
Reason: Uses the LLM to interpret context and plan multi-step actions.
Act: Executes tasks through tool integrations.
Learn: Improves through feedback loops.

Enter Strands SDK and AgentCore

You might be asking yourself how to actually do this without creating thousands of lines of boilerplate code. The answer is in Strands SDK and Amazon Bedrock AgentCore.

AgentCore delivers production ready serverless runtime, controlled state and observability. Strands SDK is the Python framework you use to build on it. Strands offers model flexibility, state management, and simple interface with OpenTelemetry for monitoring.

Here is how simple it is to create your first agent using Strands:

from strands import Agent
from strands.models import BedrockModel

# Create a Bedrock model configuration
model = BedrockModel(
    model_id="anthropic.claude-3-5-sonnet-20241022-v2:0",
    max_tokens=1024
)

# Create an agent
agent = Agent(
    model=model,
    system_prompt="You are a helpful AI assistant specialized in AWS."
)

response = agent("What are the benefits of serverless computing?")

Giving Your Agent a Brain: State and Memory

Without the ability to remember earlier interactions, an agent cannot do complex tasks. Strands handles state in three different ways:

Request State: Context just for a single interaction.
Conversation History: The back-and-forth chat history.
Agent State: Key-value storage for long-term details (like user preferences) that persists across multiple requests.

In production, you can’t merely rely on your laptop’s memory to perform session persistence. Strands provides a FileSessionManager for local testing, however it is straightforward to replace this with an S3SessionManager to store session data on Amazon S3 for distributed cloud environments.

Optimizing the Context Window

LLMs are limited in how much text they can process at once. If your talk goes on too long, the agent will crash or forget stuff. The session shared excellent conversation management ways to solve this.

A SlidingWindowConversationManager can be used to only keep the past 20 messages. Or, better yet, a SummarizingConversationManagerusing a smaller, cheaper model (such Claude Haiku) that intelligently compresses earlier messages while leaving the most current messages alone.

Equipping Agents with Tools

An agent is only as good as the tools it’s working with. Imagine you're ordering meals on Swiggy, you need the app to talk to the restaurant, the delivery guy and the payment gateway.

Strands allows you to construct regular Python functions and simply fit your agent with them with a simple @tool decorator.

from strands import Agent, tool

@tool
def check_shipping_options(zip_code: str, product_id: str) -> dict:
    """Check available shipping options."""
    # Logic to call your shipping API goes here
    return {"method": "Express", "cost": 15.00}

# Pass the tool to the agent
website_chatbot = Agent(
    model=model,
    tools=[check_shipping_options],
    system_prompt="You are a helpful shopping assistant."
)

When the user asks about shipping, the agent automatically reasons that it needs to run the check_shipping_options function, executes it, and formats the output for the user.

Scaling Up: Multi-Agent Patterns

This is where it gets super powerful. Sometimes you give an agent too much responsibility and he gets lost. Or, better yet, you can split and conquer using Multi-Agent patterns.

Workflow Pattern: A choreographed performance. A Researcher agent provides data to an Analyst agent, which provides data to a Writer agent.
Agents as Tools: The “Orchestrator” agent gets the user prompt, then determines to query a specialist Billing Agent or a Tech Support Agent.
Swarm Pattern: A very collaborative environment where an Architect, a Coder and a Reviewer send a task back and forth till it is great.

The golden rule for multi-agent systems? Always specify clear responsibilities, precise time limits, and avoid agents being locked in an unending dialog loop with each other.

Observability and Guardrails

When your agent is out in the wild, you need to know what it is doing. Strands deeply integrates with OpenTelemetry and systems such as Langfuse.

Telemetry setup provides you with three pillars of observability: Traces (end to end flow of the request). Metrics (quantitative data like token consumption and costs). Logs. You may even establish Custom Spans around your tools to monitor exactly how many milliseconds your payment processing tool took to run.

And to keep the AI safe, Amazon Bedrock Guardrails are like the bumpers at a bowling alley. They can be configured to automatically block harmful content, redact PII (e.g. credit card details) and avoid prompt injection attacks. For example, if a user asks a shopping agent about an approaching political election, the Guardrail immediately jumps in and stops the off-topic remark.

The Final Mile: Production Deployment

Long story short, you can’t deploy a local script, thus it’s pointless. Strands and AgentCore make this shockingly easy. Your agent code can be packaged as either a ZIP file (for basic Python setups) or as a Docker container (for heavy dependencies).

You just need to add an entry point decorator around your main function for deployment to AgentCore Runtime:

from bedrock_agentcore_starter_toolkit import Runtime
import boto3

agentcore_runtime = Runtime()

# Configure the deployment
response = agentcore_runtime.configure(
    entrypoint="ecommerce_agent/main.py",
    auto_create_ecr=True,
    requirements_file="ecommerce_agent/requirements.txt",
    agent_name="ecommerce_agent"
)

# Launch it!
launch_result = agentcore_runtime.launch()

When deployed, your application can safely invoke this agent hosted in the cloud with boto3.

Key Takeaways

Some things to take away from this session:

Reactive to Proactive: We're expanding beyond the standard chatbot. Agentic AI is autonomous, anticipates needs, and acts through a continuous cycle of perception, reasoning, action, and learning.
Memory is Crucial: Agents must have brains to be useful. Short-term and long-term state management is a key area to keep your context windows optimum and your agents intelligent.
Divide and Conquer: Don’t overburden one agent. Enhance the reliability and precision of routing jobs to specialized agents with multi-agent patterns like Swarm or Workflow.
Safety and Visibility: You can’t control what you can’t see. For real world production, you must trace execution timings, manage token costs, and enforce Bedrock Guardrails.

Conclusion

In the end, the difference between writing a toy AI script on your laptop and deploying a durable, production-ready AI agent is huge. But tools like Amazon Bedrock AgentCore and the Strands SDK provide a lovely bridge over that divide. Managed memory, simple tool integration, and deep observability provide us what we need to construct scalable, autonomous systems safely.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS User Group Chennai Meetup

Topic: From Build to Telemetry - Bedrock Agents with Strands SDK

Date: November 15, 2025

Also Published On

AWS Builder Center

Hashnode

Moving Beyond Vibe Coding: A Deep Dive into AWS Kiro and the AI-DLC

N Chandra Prakash Reddy — Tue, 23 Jun 2026 17:08:50 +0000

One of the best ways to keep up with the fast-moving cloud sector is, frankly, to go to community tech events. I had the privilege of attending the AWS User Group Chennai Meetup on 15th November 2025. There were loads of great topics throughout the day on all sorts of cloud technologies but one of them really changed how I look at programming in the future.

The presentation focused about using AWS Kiro, an agentic IDE for building AI-DLC (AI-Driven Development Lifecycle). As someone who writes and tests code on a daily basis, I could definitely relate to this talk. This blog is for you if you’ve ever felt AI coding tools are powerful but a bit unpredictable.

Here is everything I learned from the session, broken down step by step.

The Problem: Why "Vibe Coding" Is Not Enough

Does this sound familiar? You start an AI chatbot, describe the app you want in a few sentences, and the model spits out hundreds of lines of code quickly. The program operates. It looks good. And you feel like a magician. This is what we call "vibe coding" - coding by just defining the overall mood or idea of the application.

Vibe coding is perfect for short prototypes or weekend tweaks. But here is the thing, the instant you worry about enterprise security, team cooperation or deploy to production, it falls apart.

The speaker revealed a surprising statistic: independent research has shown that roughly 45% of AI-generated code is filled with security flaws. These include harmful injection faults and bad dependency choices.

Because the AI is only guessing off of your “vibes,” it will hallucinate APIs, software packages, and functions that do not really exist. This results in exploitable flaws and a terrible supply-chain danger known as “slopsquatting,” when malicious actors register the identities of the phony dependence the AI hallucinated.

Picture this: you're ordering food on Swiggy, but you just type in, "I want something spicy." You could end up with a great curry or a bowl of raw chillies. Without explicit parameters you are leaving the results to chance.

AWS Kiro: The Answer to Vibe Coding

The session then offered AWS Kiro as a solution to this mess. Kiro is an AWS-native, agentic IDE (Integrated Development Environment) built to address exactly these kinds of difficulties.

Instead of letting the AI run free, Kiro offers structure. It introduces spec-driven programming, routing rules and automatic hooks. In other words, it assures your AI-assisted coding is well-structured, highly auditable and truly production-ready.

The Spec-Driven Flow

Traditional AI tools go straight from your prompt to writing code. AWS Kiro implements a smThe Spec-Driven Flowarter 5-step workflow:

Idea / Prompt: You describe your initial need.
Specs: Kiro generates concrete requirements, user stories, and acceptance criteria.
Tasks: It creates an implementation checklist.
Code & Tests: The AI writes the code alongside automated tests.
Run & Integrate: Everything is deployed and connected.

Imagine your database is a library and you are adding a new wing. You don’t just bring in builders and instruct them to start building bricks based on the “vibe” of a library. First you would draw blueprints. Kiro is the architect, having you develop those plans before a single line of code is produced.

A Real-World Example: Building a Shopping Chatbot

The speaker guided us through a live example of constructing a grocery buying chatbot tied to GPT-4o during the lecture. It was fun to see the Kiro UI in action.

Kiro didn’t merely throw out Python code when the project began. Instead, it asked explanatory questions like: Which platform do you prefer ? What shopping features would you like? Already have an API key from OpenAI?

Generating the Blueprints

When the developer responded, Kiro created a requirements.md file in the EARS format (a formal means to express clearly defined system requirements). It laid out specific user stories, including letting a buyer search for products or get cooking suggestions based on dietary constraints.

From there Kiro went on to the design process. It automatically created specific markdown files for the architecture, data flow, error handling and unit testing approach for the solution. Lastly, it made a Kanban-style task board (tasks.md) to graphically track progress.

To be fair, it would take a dev team days to setup all this manually. Kiro completed it in seconds, yet with the rigor and documentation of a senior technical team.

The Secret Sauce: Hooks, Steering, and MCP

And this is when it gets interesting. AWS Kiro is more than simply a chat window, it is tightly integrated into the developer workflow with a few killer features.

Agent Hooks

Kiro features an automated Agent Hook System to remove repetitive tasks. It watches your workspace for certain occurrences (e.g. saving a file or making a code commit).

It executes a specified prompt when an event occurs. For example, the speaker displayed a hook that automatically updated the master requirements when a new file was requirements.md file to keep the documentation exactly in sync. Think of it as a smart thermostat that automatically changes the temperature when you open a window.

Project Steering

If everyone on a team codes in their own way the code base quickly becomes a nightmare . Kiro implements a steering.md file to solve this.

This is the steering wheel for the AI agent. It enforces code standards, architectural patterns and organizational principles. This enables team alignment and very consistent code generation, regardless of who in the team is using the tool.

Model Context Protocol (MCP)

Kiro can also connect to additional tools and data sources via MCP (Model Context Protocol). Looking at the mcp.json configuration file I saw how Kiro could be plugged directly into AWS documentation and give the AI agent real-time, correct context from the source.

Traditional SDLC vs. The AI-DLC

You’re probably thinking, why do we need a new tool solely to create code? The speaker made a strong case for how our existing Software Development Life Cycle (SDLC) is actually holding AI behind.

The classic SDLC (Plan --> Design --> Build --> Test --> Deploy) was built for human-driven, long-running procedures with plenty of heavy planning, meetings, and manual handoffs. Most organizations are merely adding AI to this old procedure right now. We employ AI as a narrow aid to develop a rapid function but the basic obstacles are still there.

Enter the AI-Driven Development Lifecycle

“To truly unlock the potential of generative AI, we need to rethink the whole lifecycle. The AI-DLC is bringing AI into the heart of the partnership, not as a bolt-on sidekick.

The core working pattern is a continuous, high-speed loop:

The AI creates a plan.
The AI asks clarifying questions.
Humans validate and make the critical decisions.
The AI implements the validated plan.

New Language and Rituals

The process is so substantially faster that standard Agile language is expanding under the AI-DLC

Use highly intense "Bolts" (hours or days) instead of long "Sprints" (weeks).
Massive “Epics” are now just “Units of Work.” Instead, the new methodology promotes “Mob Elaboration” and “Mob Construction” over developers working in isolation, where the entire team comes together to collectively make technical decisions and solve complicated business challenges around the AI’s output.

The Practical Benefits of AI-DLC

Bottom line: The shift to this spec-driven, AI-first architecture has huge benefits for organizations and developers alike.

Massive Velocity: Requirements, design and coding requirements, design and coding 5x-20x faster. Work that took weeks is compressed into hours.
Higher Quality: Kiro needs constant explanation, so the end result is much closer to the genuine business goal. It also enforces organizational norms throughout the board.
Better Developer Experience: Less time spent on boilerplate programming means more time for innovative issue solving and genuine business value.

Key Takeaways

I walked out of this seminar with a new perspective on where cloud development is headed. Here are the primary things I got from the talk:

Architecture Metrics Matter First: You can’t build a fantastic AI application on flawed architecture. Event-driven patterns such as Amazon EventBridge and AWS Step Functions are still important for developing robust systems.
Specs are the New Code: In an AI-driven environment, your ability to write precise, organized specifications is more important than your ability to type syntax. Specs ground AI and keep it correct, and tools like Kiro prove it.
Security Cannot Be an Afterthought: Almost 50% of code produced by AI is vulnerable, hence for production systems we need automatic guardrails like steering files and tight review loops.
The Developer's Role is Shifting: We’re moving away from just syntax writers and heading toward system orchestrators, architects and strategic decision-makers.

Conclusion

In the end, AI is transforming software development at a basic level, but if we’re serious about building safe, enterprise-grade systems, we can’t just rely on raw “vibe coding.” The transition from traditional SDLC to the AI-DLC is a tremendous leap in the delivery of value by the teams.

AWS Kiro gives you the framework, the guardrails and workflows you need to treat AI as a true and dependable engineering partner. Programming is not about typing faster. It's about thinking clearer. And directing intelligent agents with accuracy.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS User Group Chennai Meetup

Topic: Moving Beyond Vibe Coding: A Deep Dive into AWS Kiro and the AI-DLC

Date: November 15, 2025

Also Published On

AWS Builder Center

Hashnode

When Systems Listen: Event Driven Architecture

N Chandra Prakash Reddy — Sun, 21 Jun 2026 08:48:06 +0000

To master real-time AWS applications, focus on events rather than handling a constant stream of requests.

Attending a full-day tech conference can be overwhelming with all the information you take in. On 01 November 2025, I got to attend the AWS Student Community Day in Tirupati. The event featured many great sessions about different parts of cloud computing.

The last session of the day really changed how I think about building modern applications. Poobalan gave a talk called "When Systems Listen: Event Driven Architecture." Even though it was late in the day, everyone was energized, and the ideas he shared were very practical.

I took lots of notes and snapped photos of the slides to capture everything. Now, I’ll share what I learned about Event-Driven Architecture (EDA) and explain why it’s important for developers today.

The "Are We There Yet?" Problem

Before we can see why Event-Driven Architecture matters, it helps to look at how traditional systems talk to each other. Poobalan kicked off the session with a familiar example: ordering food online.

Think about ordering your favorite meal on a food delivery app like Swiggy or Zomato. After you place your order, you watch your screen, waiting for updates. With traditional request-driven systems, your app would keep asking the restaurant's server, "Is it ready yet? How about now? Is it cooking now?"

Does this sound familiar? This is known as polling. Imagine being on a road trip with a child in the backseat who keeps asking if you are there yet. In software, this kind of constant checking uses up resources, puts extra strain on servers, and can cause annoying slowdowns. In the presentation, a funny meme showed a guy sitting alone on a park bench, which summed up the pain of always 'checking for updates.'

You might be wondering, isn't there a better way to handle this?

Enter Event-Driven Architecture

Here’s where things get interesting. Event-Driven Architecture changes the usual approach. Instead of components always checking in with each other for updates, the system follows the old Hollywood saying: "Don't call us, we'll call you."

In an EDA setup, components communicate by sending and receiving events. When something happens in your application, it sends out an event. Other services that are interested in that event listen for it and respond as needed.

The Anatomy of an Event

The slides explain that an event is just a change in your system’s state. There are also three main rules that define what makes something a true event:

Past tense: Events should describe something that has already happened. For example, use names like "Order Created" or "User Signed Up" instead of "Creating Order."
Immutable: After you publish an event, you will not be able to make any changes to it.
Decoupled: The system that publishes the event does not know who is consuming it, and it is not concerned with this information.

The Three Musketeers of EDA

The system that sends out the event does not know who will receive it, and it does not need this information.

Event Producers: These systems or services create events whenever something changes, such as when a customer clicks the checkout button on an online store.
Event Router/Broker: You can think of this part like a post office. It takes in events and sends them to where they need to go.
Event Consumers: These services wait for certain events so they can respond and do their jobs, such as sending a confirmation email after an order is placed.

Traditional vs. Modern: The Showdown

If you usually build request-driven applications, moving to event-driven architecture means thinking differently. In the session, a clear side-by-side comparison helped show exactly how the two approaches differ.

In a Request-Driven setup, communication happens in real time. The client sends a request and waits for a response. Since the systems are closely linked, if one service is slow, everything else slows down too. This setup does not scale well, and heavy traffic can quickly lead to bottlenecks across the system.

In EDA, communication happens asynchronously. When a client sends an event, it can continue working right away. The systems are loosely coupled, so responses do not block the process. This setup makes it possible to handle millions of independent events without slowing down the main application thread.

Here’s the main idea: request-driven systems are like calling someone and expecting them to answer right away. EDA is more like sending a letter; the person can read it and respond when it suits them.

Why We Should Care

Switching to an event-driven model brings several important benefits for today’s cloud applications:

Loose Coupling: Since services don’t need to know about each other, different teams can develop, deploy, and update them independently.
Real-Time Responsiveness: Systems can respond to changes the moment they happen, which allows for instant decision-making.
Resilience: If your email notification service goes down, your checkout service keeps working. It continues sending "Order Placed" events, which can be replayed when the email service is running again.
Cost-Effective: When you remove the need for constant polling, you can greatly lower your operational and computing costs.

Building EDA on AWS

At AWS Student Community Day, Poobalan moved on to talk about the main AWS services that support Event-Driven Architecture. These are the tools you use to build these systems.

Amazon EventBridge

This service is a serverless event bus that links your applications. It works like an event traffic controller by taking in events, checking them against rules you set, and sending them to the right places.

Amazon SNS (Simple Notification Service)

This service uses a publisher/subscriber model for messaging. Its main job is to broadcast messages. Picture it as 'one message, many listeners.' If you need to notify several backend services at once when an event happens, SNS is the right choice.

Amazon SQS (Simple Queue Service)

SQS is a message queue. It stores messages safely until the coSQS is a message queue that keeps messages safe until your services are ready to handle them. Imagine your database as a library, and 500 people try to check out a book at the same time. SQS works like a message waiting room, putting everyone in order so the librarian is not overwhelmed and no requests are missed.

Seeing it in Action: The Ride Booking App

Learning theory is helpful, but seeing it in action really helps it sink in. In the presentation, we watched a live demo with a made-up taxi ride-booking app.

We looked at the Amazon EventBridge console on the screen, which showed different routing rules. There was one rule named ride-to-driver-matching and another called driver-to-notification.

When someone clicked "Book a Ride" on the app, it triggered an event. We saw the Service Control Panel and the Ride Status Dashboard update right away. Thanks to event-driven architecture, the Driver Matching Service and the Notification Service worked on their own without depending on each other.

The dashboard displayed progress bars moving from "Ride Requested" to "Driver Assigned" and then to "Confirmation Sent." If the Notification Service went down for a bit, the app kept running. The driver was still assigned, and the notification just waited in a queue until the service was back online to handle it.

Is EDA the Silver Bullet?

EDA is impressive, but it is not the right solution for every project. The speaker made it clear when to use it and, just as importantly, when to avoid it.

When to use EDA: Event-driven design works best when you need real-time responses, high scalability, and flexibility through decoupling. It is ideal for complex systems with many producers and consumers, or when integrating different platforms.

When NOT to use EDA: Setting up event buses and queues can make your architecture more complex. If you are working on a simple CRUD application, a small prototype, or an app that needs strictly ordered and immediate responses, it is better to use traditional request-driven architectures. In some cases, a simpler approach is best.

Doing it Right: Best Practices

If you are ready to start using EDA, here are some important best practices to help keep your systems stable:

Idempotency: This idea sounds complex, but it is simple. Make sure your consumers can handle processing the same event more than once without causing errors or creating duplicate records.
Error Handling: Networks sometimes fail. Set up retries for temporary issues, and use Dead Letter Queues (DLQs) to store events that cannot be processed, so you can review them later.
Scalability: Set up your event channels with partitioning and load balancing. This way, your system can handle more traffic smoothly when demand increases.
Monitoring & Logging: Since events move through different systems at different times, debugging can be hard. Use observability tools to track and visualize event flows, so you always know what is happening in your system.

Key Takeaways

Here are the main ideas I’m bringing back to my own projects after this packed session:

Stop the polling: Switching from always checking on progress to a setup that just waits for notifications can save a lot of computing power.
Decoupling is a superpower: If your backend services don’t depend on each other, teams can build, scale, or even have issues without affecting the whole app.
AWS does the heavy lifting: Tools like EventBridge, SNS, and SQS work like a post office and waiting room, so it’s easy to route and store messages safely.
Complexity isn't always the answer: Even though I enjoyed the session, EDA isn’t always needed. Simple apps don’t need this much engineering, and that’s totally fine.

Conclusion

In short, going to the last session at AWS Student Community Day Tirupati was a great way to wrap up an already fantastic event. Poobalan managed to take a complex architectural idea and explain it using real-life examples.

Event-Driven Architecture helps our systems act more like the real world by reacting to events as they happen, instead of needing constant updates. It takes a new way of thinking, but the benefits in scalability, cost savings, and resilience make it worth learning.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Student Community Day Tirupati

Topic: When Systems Listen: Event Driven Architecture

Date: November 01, 2025

Also Published On

AWS Builder Center

Hashnode

The Future is Conversational: Analyzing Cloud Networks with GenAI

N Chandra Prakash Reddy — Tue, 16 Jun 2026 06:18:17 +0000

I had the fantastic opportunity to attend the AWS Student Community Day in Tirupati on November 1, 2025. The sessions were great and covered a wide spectrum of cloud subjects but one particular speaker particularly caught my attention.

The session was titled “The Future is Conversational: Analyze Cloud Networks with GenAI” and was led by Yeshwanth L M. Yeshwanth is a multi-talented individual in the tech community. He is a Solutions Engineer at Confluent, an active AWS Community Builder, a Microsoft Certified Trainer, an IEEE Volunteer, and the creator of the YouTube channel “Tech With Yeshwanth.”

His lecture handled a big pain point for cloud engineers and made it easier with Generative AI. This is what I took away from his highly entertaining talk.

The "Khatabook" Problem: Managing a Giant Network

Yeshwanth opened the discussion with a fantastic everyday analogy to explain cloud networking. Imagine your AWS network as a huge college hostel and you as the warden.

In this analogy, you have 10,000 pupils (your cloud servers), thousands of dormitories (your applications), and one main entrance (the Internet). As the warden your major job is to be aware of who is coming in and out of the main gate and to make sure the pupils are safe.

Let’s face it, keeping track of 10,000 students isn’t easy. At the main gate, your security guy logs everything into a big notebook. In the realm of AWS, this logbook is called a VPC Flow Log.

Now picture this logbook with 100 million entries a day! This leads to what Yeshwanth aptly termed the “Khatabook” (logbook) dilemma.

The Warden's Nightmare

This vast amount of data creates a terrible scenario. Suppose the college principal asks you a very precise inquiry. “Did anyone try to sneak into the girls' hostel (a secure server) at 2 AM last night?”

In old school environment your job? You have to go through 5,000 pages of the logbook with a pen and locate the answer. This is the “Old Way” that cloud infrastructure was. Technically this entails building complex, very precise SQL queries to sift through millions of logs.

Does this sound familiar? It’s sluggish. It’s unpleasant. And by the time you finally compose the appropriate query, get the answer, the intruder is already long gone.

The "What If" Moment

Here’s the thing… what if you didn’t have to create those complicated queries? What if you could just naturally ask a question?

Picture yourself pulling out your phone, texting an assistant: “Hey, show me all ‘REJECTED’ entries after 1 AM." Or, more college-campus relatable: “Which student (IP address) is ordering the most Swiggy or Zomato orders?”

This simple transition from writing hard-coded code to having a natural discussion is exactly where Generative AI comes in to revolutionize the game entirely.

Meet the "Super-Smart Warden": Amazon Bedrock

Yeshwanth’s answer to the “Super-Smart Warden” of the AWS ecosystem was Amazon Bedrock.

Amazon Bedrock is a fully managed solution that brings Generative AI into your cloud environment. Imagine giving your hostel a super brain It’s like having a Ramanujan, a mathematical genius, on your administrative staff. Instead of you scanning the book with a pen, this genius can read all 5,000 pages of your logbook in one second and just tell you the answer.

Traditional vs. Intelligent Architecture

To see what a major shift this is, we have to look at the warden's "Filing System".

The Traditional Architecture:

In a normal legacy arrangement, network traffic passes via your Virtual Private Cloud (VPC) and gets captured in VPC Flow Logs. The data is then dumped onto storage such as an Amazon S3 bucket or Amazon CloudWatch. You would normally use a technology like Amazon Athena to query the S3 data and then develop a visual Dashboard to interpret this. It works but it’s a lot of heavy lifting, ongoing maintenance and a lot of manual query building.

The Intelligent Architecture:

The current technique streamlines this pipeline entire. Your VPC logs continue to flow to S3 and CloudWatch. But the data is immediately streamed into Amazon Bedrock, not through Athena and constructing fancy dashboards. You just ask the AI “Is there any strange activity from someone I don’t know?” and it answers in a conversational manner.

The Magic Trick: How It Actually Works

You may wonder how it works behind the scenes? Is this a magic? Yeshwanth broke down the technical reality into a four-step approach.

Step 1: Getting the Logbook Pages

First of all, the system needs raw data. For example, a script calls a function get_flow_log_data. It is precisely the same as going up to the security officer and asking him for all the logbook entries from the last 24 hours. It extracts the raw text messages directly from Amazon CloudWatch Logs.

Step 2: Pre-Processing in Python

Even before you talk to the AI, the script gets clever. It executes a Python function called analyze_flow_logs_summary that uses the Pandas package. This is where you take the raw data and make a high level summary, automatically finding things like total records, top IPs, total bytes moved, etc.

Then it calls a method called format_flow_logs_for_bedrock to provide a nice, legible text "printout" of the first few log entries. Think of this phase as cleaning up your desk and summarizing your notes before your boss comes to see a report.

Step 3: Building the "Super-Prompt"

This is where things become fascinating. This is where the true magic happens. We don’t just ask a plain question to the AI. We feed one giant dossier of information into Amazon Bedrock, all at once. This “Super-Prompt” has three primary components:

The Role: We give the AI a persona by stating, "You are an AWS network expert."
The Summary: We provide the Pandas data we just created: "Here is a summary I already made: {Total Records: 5000, Top IP: 1.2.3.4...}"
The Raw Data: We feed it the actual log entries: "Here are the actual log entries: {1. 10.0.0.1 -> 10.0.0.2 [REJECT]...}"

Step 4: The "In-Context" Answer

Finally, we append the real question to the very end of this huge prompt. We add for example: "...Now, tell me this: 'Are there any suspicious activities?

Bedrock reads the pre-computed summary and the raw logs we just gave it. It uses that current context to locate the solution. Bottom line. The AI is not going out and searching a regular back-end database. It’s basically reading and understanding the huge chunk of material you just given it and giving you a human like answer.

Live Demo & Beyond Security

After discussing the principle, Yeshwanth walked into the AWS Console for a live demo to "catch some intruders" and how this architecture parses flow logs in real time. Seeing the AI immediately evaluate complex network traffic was super cool.

But the main point of the session’s message was much broader than merely AWS security. This log analysis architecture is just the beginning, the future is conversational. You can execute this very same four step “magic trick” on just about any data generating domain:

For a Cricket Analyst: You could feed ball-by-ball IPL data to an AI and ask, "What bowling pattern makes Virat Kohli get out most often?"
For a College Admin: You could upload raw campus Wi-Fi logs and ask, "Which classroom's Wi-Fi is the slowest?"
For Developers: You could take your user-click logs and ask, "Which part of my app is making users angry?"

Key Takeaways

If you only remember a few things from this session, here is what you need to know:

Goodbye to Complex Queries: You don’t have to write complex SQL anymore, nor search through millions of lines of text manually. With GenAI you can just ask natural questions to examine large datasets.
Context is Everything: The answer to making this work is not just asking a smart question. It’s creating a “Super-Prompt” that first sends Amazon Bedrock your summary data and raw logs so it gets the precise context it needs to offer a great answer.
Endless Possibilities: Yeshwanth used AWS network security for his example, but this conversational design works for anything. GenAI can be your analyst on call, whether it’s cricket statistics, college Wi-Fi, or app user behavior.

Conclusion

At the end of the day, going from writing rigid code syntax to having a simple discussion with your infrastructure is a big game shift. Generative AI is not just a pretty new toy to generate emails. It is a very practical, powerful technology that can be the ‘super-smart warden’ for your most complex operational data.

Attending this session at AWS Student Community Day was eye opening. It is getting easier fast to grasp complex systems. If you haven't yet played around with Amazon Bedrock or GenAI for your own data, I advise you to give it a try. Write a simple script, give it some logs and see what happens. The future is indeed conversational ! And all it takes to get started is understanding how to ask the correct questions !

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Student Community Day Tirupati

Topic: The Future is Conversational: Analyzing Cloud Networks with GenAI

Date: November 01, 2025

Also Published On

AWS Builder Center

Hashnode

Demystifying Terraform: Bridging the Gap Between Infrastructure and Cloud

N Chandra Prakash Reddy — Mon, 15 Jun 2026 16:28:10 +0000

Let’s face it, tech events are thrilling, but can also be quite overwhelming. I went to AWS Student Community Day on 1st November 2025 in Tirupati. There were some great sessions all day long on different elements of cloud computing. But there was one particular session that had me totally immersed.

Senior Cloud Engineer & AWS Community Builder Keerthivasan Kannan took the stage to discuss about ‘Bridging IaC and Cloud’. He took Infrastructure as Code, a subject that could be scary, and made it consumable and logical.

I took notes and photos throughout his talk so that I could share these ideas. If you are a student or an aspiring cloud enthusiast, this blog will help you understand how you may control the cloud resources by using the code.

The Cloud Management Problem

If you’ve ever utilized a cloud provider, you undoubtedly started by logging in to the web portal. You went through menus, choose regions, set up networks and ultimately started a server. Does this sound familiar?

Manual clicking is fantastic for learning, but a nightmare in the real-world. Imagine you have to build up fifty servers, apply sophisticated security rules and reproduce that same setup for a testing team. Doing that manually is slow, frustrating and prone to human mistake.

This is where the idea of Infrastructure as Code, or IaC, comes in. You don’t click buttons, you just write simple config files that tell you exactly what you want. You provide those files to a tool and it will create the infrastructure for you.

Enter Terraform

Terraform was introduced by Keerthivasan as the IaC tool of choice in the session. But what's so special about it?

Not Just for One Cloud

You might be wondering if knowing Terraform limits you into one platform. The slides were clear: Terraform supports multi-cloud environments. It’s not AWS-only. It works with Google Cloud, Azure and dozens of other providers.

It's like a universal remote control. You don’t need a separate remote for the TV, the sound system, and the air conditioner, you use one standard remote to operate everything in your house.

The Core Workflow: Terraform Stages

Here's the thing: Terraform operates in a very predictable, logical sequence. The session broke this down into four main stages.

Init (Initialize)

Before you can build anything, Terraform needs to prepare its workspace. The init stage downloads the necessary plugins to talk to your specific cloud provider. Imagine you are about to cook a new recipe; this step is like going to the grocery store to gather all your specific ingredients.

Plan

This is your safety net.” When you run a plan, Terraform examines your code and informs you exactly what it’s going to produce, alter, or destroy. It doesn’t affect anything for now. This is like looking over the blueprint with your contractor before the concrete gets poured.

Apply

Run apply when you approve the proposal. That’s when Terraform really speaks to the cloud and makes your resources. This is when you really turn the stove on and start cooking (in our cooking example).

Destroy

When you don’t need the infrastructure anymore, the destroy command tears it all away cleanly. It’s critical for saving money on cloud bills after a project finishes.

Remembering the Past: The State File

To be fair, managing infrastructure requires a good memory. Terraform needs to know exactly what it has created so it can manage it in the future.

Tracking Changes

Terraform maintains a record of your infrastructure in what is called a “State file.” This document ties your code to the real-world resources that are running in the cloud.

Think of your state file as like a save file in a video game. When you quit the game and return tomorrow, the save file remembers which level you are on and what stuff you collected. Without that, you'd have to start from scratch.

Cloud Storage

The slides stressed that this state file should be stored in Cloud Storage, not on your local laptop. If you are working in a team , everyone has to have the same " save file " , so you do not mistakenly overwrite each other .

Keeping Things Organized: File Structure

As your cloud environment grows, your code will too. Keerthivasan shared best practices for structuring Terraform files.

Standardizing Files

You don't just write a huge wall of code . You break it down in logical files. The presentation also showed a typical directory for an EC2 (virtual server) deployment:

providers.tf: Tells Terraform which cloud to connect to (like AWS).
variables.tf: Stores configurable values so you can easily change them later.
locals.tf: Defines local, reusable names or tags within the module.
output.tf: Displays important information after the build finishes, like the server's IP address.

Real-World Architecture

We also viewed a practical architecture diagram of a website domain (www.mbu.asia). The diagram demonstrated Terraform provisioning an S3 bucket for storage, moving traffic through a network architecture, and ultimately arriving to EC2 instances. Properly structured files make complex setups like this much easier to manage.

Building with Blocks: Modules and DRY

This is where it gets interesting. As developers, we hate doing the same work twice.

The DRY Principle

DRY is an acronym for "Don’t Repeat Yourself. The workshop taught how to do this with Locals and Variables. If you need to apply the same “Project” tag to fifty resources, you don’t write it fifty times. You define it once as a local or a variable and then refer to it everywhere.

Terraform Modules

If you find yourself writing the same code for an S3 bucket or a Security Group over and over you can construct a Module .

Think of a module like buying a cake mix. Instead than weighing out flour, sugar and baking powder every single time you want to bake, you just grab the mix, add water and you are good to go. The presentation illustrated folder structures splitting ec2, s3 and sg (security groups) into reusable modules that may be called with a few lines of code.

Advanced Magic: Variables, Loops, and Logic

To make your infrastructure truly dynamic, Terraform offers several advanced features.

Variable Validation

Sometimes you want to restrict what inputs are accepted. The slides indicated a variable block for “environment” with a strict constraint. It did validation to ensure the environment name was just "dev", "staging" or "prod". If a user inputs “testing”, Terraform will error out.

Count vs. For_each

When you want to create multiple resources, you have two main options.

Count: The slide presented an example of Web Servers creation with count = 3. If you want exact replicas, this is great. Imagine ordering three simple pizzas the same for a party.
For_each: This example shows how to create different IAM users using a map. This is best if each object has unique qualities. Imagine you are ordering meals on Swiggy for three pals, one wants a burger, the other wants pasta and the third wants a salad. For_each handles those special orders well.

Dynamic Blocks

Security Groups control what network traffic is allowed in and out. The presentation included “Dynamic blocks” that automatically create multiple entrance rules. Instead of hardcoding each and every port rule, Terraform iterates over a list and dynamically constructs the rules.

Built-in Functions

Terraform is not simply resource declaration. It can conduct math and string manipulation. The functions slide was an overview of numeric, string and collection functions. One simple example provided was the ceiling function. Ceil(5.1) immediately rounds up to 6.

Conditional Statements

Finally, we looked at conditionals. The syntax shown was condition ? true_val : false_val. In other words, it is a simple decision-maker. The slide's example checked if a variable was empty. If it was empty, it assigned a default value; otherwise, it used the provided value.

Key Takeaways

If you are looking to start your journey with Terraform and IaC, here are the core lessons I picked up from the session:

Automation is Non-Negotiable: Moving away from manual console clicks to code isn’t just a “pro” move, it’s important for consistency and speed.
Trust the Workflow: Always follow Init -> Plan -> Apply The “Plan” stage is your best buddy since it prevents costly or damaging mistakes from happening.
The State File is Sacred: Think of your state file as a treasure map. You lose it, or corrupt it, and Terraform doesn’t know what it built anymore. Keep it on the cloud (like an S3 bucket) and not simply on your laptop.
Modules Save Time: Don't write the same code twice. Create modules that are “blueprints” of common resources like servers or databases that you can install instantaneously in new applications.
Logic Makes Infra "Smart": With variables, loops (count and for_each) and conditional expressions your infrastructure can be flexible to new needs without you having to rebuild the entire code base.

Conclusion

AWS Student Community Day at Tirupati was an enlightening event. There were a lot of good lectures during the day on everything from AI to Security but this one on “Bridging IaC and Cloud” really caught my eye because of how practical it was.

When you first look at Infrastructure as Code it can feel like a mountain to conquer. But watching everything broken down into basic files, logical steps and reusable modules made it feel much more like playing with Lego blocks than building difficult software.

Terraform is a skill that will serve you well whether you are just starting your cloud journey or looking to scale a project. My best advice? Don’t just read about it: install Terraform, get a free-tier AWS account, and try deploying your first S3 bucket using the apply command. There’s no better way to learn than to do it!

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Student Community Day Tirupati

Topic: Demystifying Terraform: Bridging the Gap Between Infrastructure and Cloud

Date: November 01, 2025

Also Published On

AWS Builder Center

Hashnode

From MLOps to LLMOps: A Practical AWS GenAI Operations Guide

N Chandra Prakash Reddy — Fri, 03 Apr 2026 16:12:34 +0000

The vibe at AWS Student Community Day Tirupati on November 1, 2025, was different from what I thought it would be like. There were lots of students, cloud fans, and builders in the room. They were all there to learn, meet, and geek out about AWS. Throughout the day, there were several classes, and each one added something new.

One lesson, though, made me sit up and pay more attention. Raghul Gopal, a Data Scientist and AWS Community Builder (ML), walked up to the stage to talk about something that most people don't think much about: how do you run AI models in real life? Not just make them on a laptop and be happy about it; consistently test, watch, and scale them.

"Generative AI Operations: FMOps, LLMOps Integration with MLOps Maturity Model" was the title of the talk. When it was over, I had a whole new perspective on the AI/ML lifecycle on AWS.

The Question That Kicked Everything Off

"AWS gives you everything in one place to build ML models," Raghul said to start the talk, and it really hit the mark. But are we really using it right in production?"

Sense a pattern? A model can be trained by many teams. It's a whole different task to get that model to work reliably for a lot of real users.

To put it another way, making a great meal at home is one thing. It takes a lot of different skills to run a restaurant kitchen that feeds hundreds of people every day without any problems. That's what this meeting was all about.

What "ML in Production" Actually Means

Before getting into answers, the session gave us a really helpful list of questions that can be used as a litmus test to see if your machine learning setup is really ready for production:

Are your model's features (the pieces of data it uses to make predictions) kept separate and tracked correctly?
Is your model that you learned kept in a model repository or registry?
Is the model being watched all the time to make sure it keeps giving correct answers?
Is model lineage being kept? This is a list of which data made which version of the model.
Are there CI/CD pipelines (automated delivery systems) that move code from development to pre-production to production, with approval steps that need to be manned by manual?
Is testing done automatically in every environment?
Does ETL (Extract, Transform, Load) automatically load data so that machine learning engineers can start working on projects without haThe event also had a number of other great sessions, such as ones about cloud design, hands-on demos, and more. But this one helped me learn how to organise my thoughts in a way that I will use in all future AI projects.ving to wait for data teams?

There are a lot of people like you who answered "not really" to most of those questions. That's exactly what MLOps is meant to fix.

The Three "-Ops" You Need to Know

Let's be honest: the words can be hard to understand. It's simple like this:

MLOps (Machine Learning Operations): The process of putting standard machine learning solutions into production in a smart way. Examples include fraud detection models, recommendation systems, and churn prediction.
FMOps (Foundation Model Operations): Massive AI models like Claude or Titan are trained on terabytes of data with billions of parameters. This is an extension of MLOps for Foundation Models. FMOps includes use cases for making text, images, music, and videos.
LLMOps (Large Language Model Operations): A part of FMOps that is used to operationalise Large Language Models. This is the technology that makes chatbots, writing helpers, and coding tools work.

Imagine three rings stacked on top of each other. MLOps is the outer ring, FMOps is inside it, and LLMOps is in the middle. It doesn't matter what kind of AI model you run, all three work the same way.

The MLOps Maturity Model: Four Levels

Now things really start to get interesting. Raghul showed a four-level MLOps Maturity Model, which is a plan for how teams move from small tests to using machine learning on a large scale. It's kind of like getting better at a video game.

Level 0 - Initial Phase: Experiments and Ideas

At this point, data scientists are just looking around. To make and test models, they use Adobe SageMaker Studio (AWS's cloud-based ML IDE) or local tools like VS Code and PyCharm. This is what the technology stack looks like:

Amazon SageMaker: Core ML platform with Data Wrangler (data prep), Pipelines (automation), Feature Store, and Clarify (bias detection)
Amazon S3: Stores your raw ML training data
AWS Glue: ETL service - cleans and transforms data before feeding it to models
Amazon Athena: Run SQL queries directly on data sitting in S3
AWS Lambda: Trigger automated jobs and workflows
The event also had a number of other great sessions, such as ones about cloud design, hands-on demos, and more. But this one helped me learn how to organise my thoughts in a way that I will use in all future AI projects.
Code Repository: AWS had its own CodeCommit, but now most people use GitHub or Bitbucket to store and track their work.

That's fine; everything here is done by manual and by exploration. The beginning of every fully developed machine learning system.

Level 1 - Repeatable Phase: Automating the Workflow

From doing runs by manual, the team now goes on to automated pipelines. You don't have to re-train a model by manual every time because SageMaker Pipelines can do the data preparation, training, evaluation, and packaging for you. The SageMaker Model Registry is a central list of all your model versions that gets updated when new models are trained.

"I trained this once" became "every training run is tracked, versiThe event also had a number of other great sessions, such as ones about cloud design, hands-on demos, and more. But this one helped me learn how to organise my thoughts in a way that I will use in all future AI projects.oned, and reproducible."

Level 2 - Reliable Phase: Adding the Safety Net

This is the quality gate before going live. You introduce:

Automated testing: Unit tests, integration tests, and evaluation metrics that are specific to machine learning are all run immediately.
CI/CD Pipelines Using AWS CodePipelines and AWS CodeBuild to move code from development to pre-production to production, with approval steps that need to be manned by manual.
Different testing strategies based on how data arrives:
- Batch requests: Tested via Lambda and S3
- Real-time requests: Handled through Amazon API Gateway
- Streaming requests: Managed with Kafka and Amazon MSK

To be fair, this level demands real engineering discipline. But it's what separates a prototype from something you'd stake your business on.

Level 3 - Scalable Phase: Multi-Team, Enterprise-Scale

Everything from Level 2 is multiplied across different teams and machine learning solutions at the same time in the last level. New things added here:

Multiple data sources: NoSQL databases like DynamoDB and DocumentDB for different team needs
IAM (Identity and Access Management) to manage roles and permissions at scale
CloudFormation or Terraform for Infrastructure as Code - your entire environment defined in code, replicable in minutes
Your team can choose to use GitHub Actions or Jenkins instead of AWS CodePipelines if they already know how to use those tools.

What is the goal at this level? From idea to production in days instead of weeks, and use more than one option at the same time.

Making the Leap: MLOps → LLMOps

When you have a strong base in MLOps, moving on to LLMOps is easier than it sounds. The slide made it clear: "You can operationalise your basic LLM use cases from one environment to the next."

The ideas behind Dev, Pre-Prod, and Prod environments, CI/CD pipelines, manual approvals, and automated tests are all the same. Now you're working with Foundation Models instead of the old ML models, which is different. They're the building blocks you use to build on top of your MLOps skills.

Initial LLMOps: Picking the Right Foundation Model

This is where lots of teams get stuck. How do you pick from the dozens of LLMs that are out there? The lesson gave a framework that could be used right away.

Step 1: Know Your Use Case First

Make sure you know what you need before you choose a type. The things to look at are:

Open source vs. proprietary?
Commercial license compatibility
Model size: Small Language Model (SLM) vs. Large Language Model (LLM)
Speed and latency requirements
Context window size - how much text the model can process at once (measured in tokens)
Quality of the training dataset and how it applies to your area
Is the model fine-tunable with your own data?

Step 2: Navigate the Speed-Precision-Cost Triangle

The truth is that you can't have everything. Raghul showed this with a triangle that showed three objectives that were at odds with each other:

High speed → smaller model → lower precision → lower cost
Higher precision → larger model → lower speed → higher cost

In the case on the slide, three Foundation Models were put side by side. FM1 had the highest accuracy (5/5) but also the highest cost. FM3 was less expensive ($$), but it wasn't as accurate. When price was the most important factor, FM2 was chosen because it had the best mix of accuracy (4/5) and low cost ($). The best choice is always based on which triangle point is most important to you.

Step 3: Build a Prompt Catalog and Evaluate Systematically

Don't just pick a model and hope it works. The recommended process:

Prompt Engineers make good evaluation questions by following organised rules like CORS or Anthropic's instructions.
In a Prompt Catalogue, you can store those prompts. It's kind of like a Feature Store, but for prompts. With version control turned on, DynamoDB works well here.
GenAI Developers shortlist the top 3 Foundation Models based on those prompts
You can do structured evaluations in one of four ways, depending on the facts you have:

Accuracy metrics (when labeled data exists with discrete outputs — e.g., classification)
Similarity metrics like ROUGE or cosine similarity (for open-ended text outputs)
Human in the Loop (HIL): Using tools like Amazon SageMaker Ground Truth, human judges score model outputs by manual against set criteria.
LLM-as-judge: Feed outputs to a trusted, reliable LLM and have it rate the response with a score and explanation

The result is a clean evaluation scorecard, which means that you chose your model based on facts instead of your gut.

Building and Deploying Your LLMOps App

What do you do now that you've picked your LLM? Building the real app around it is the last step:

Frontend: Django, Flask, Streamlit (highly recommended for quick and clean prototypes), or React
Backend / LLM Provider: Amazon Bedrock, SageMaker JumpStart, or HuggingFace - depending on your model choice
Load Balancing and Auto Scaling to handle real-world traffic without hiccups
The same Dev → Pre-Prod → Prod pipeline from MLOps applies - always test your LLM in Pre-Prod before exposing it to end users

The architecture changes based on whether you're delivering at the edge or through a centralised group.

Key Takeaways

After this lesson, a few things really stuck with me:

Building a model is the easy part. Running it consistently in production, testing it, keeping track of its history, and being able to do it again is the real engineering work.
The MLOps maturity model is a journey, not a checklist. You can start at Level 0 if that's where you are now. You get to the higher levels bit by bit.
LLMOps is MLOps with a GenAI lens. You're a lot closer to LLMOps than you think if you already know how MLOps works.
Model selection should be data-driven. You don't have to guess or worry about which LLM to choose because of the prompt catalogue and organised evaluation method.

Conclusion

In the end, Raghul's talk made it clear that having the tools isn't enough; what counts is how you use them. From SageMaker to Bedrock to CodePipelines, AWS gives you a very full set of tools. But even the best tools can't fix a broken process if you don't think about things like testing, tracking, and being able to do the same thing again.

If you're a student just starting to learn machine learning, a developer looking into GenAI, or an engineer building real systems at work, you need to understand this operational layer. This is what sets people who play with AI apart from those who ship AI. The talk at AWS Student Community Day Tirupati taught me that there isn't as much of a gap between the two as most people think. You have to get on that growth curve somewhere and keep going up.

The event also had a number of other great sessions, such as ones about cloud design, hands-on demos, and more. But this one helped me learn how to organise my thoughts in a way that I will use in all future AI projects.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Student Community Day Tirupati

Topic: From MLOps to LLMOps: A Practical AWS GenAI Operations Guide

Date: November 01, 2025

Also Published On

AWS Builder Center

Hashnode

LamRAG: AI-Powered Feedback Analysis Using Amazon Bedrock

N Chandra Prakash Reddy — Sun, 29 Mar 2026 15:35:02 +0000

There were a lot of great talks at AWS Student Community Day Tirupati on November 1, 2025. But the one that really jumped out was "LamRAG: From data to constructive insights using Amazon Bedrock" by Rahul Kumar and Gokul Jangam.

It wasn't a normal slide-and-talk presentation. It was a live, step-by-step tour of a real product they made called Feedbackly, which is a platform for managing feedback. They showed how they improved it over time using Amazon Bedrock. The session was set up perfectly, with levels that built on each other. When it was over, I got a whole new way of looking at what generative AI on AWS can do.

Let me break it all down for you.

The Problem: Feedback Chaos and the 10/10 Trap

You know how messy it can get if you've worked on a team that does sprints. Many projects. A lot of project managers. Different schedules for sprints. Different approaches to get feedback from peers. There is no one spot to keep track of it all.

Feedbackly was made to solve just that: a single system for managing projects and getting peer input from team members every sprint. It's like a notebook that everyone in your engineering department can use to keep track of comments from every sprint.

But then there was a new problem that seems quite familiar: everyone started giving 10 out of 10 ratings. Everyone is "great." Everyone "did better than expected." The feedback loses its meaning. Not important. Not useful for real conversations about performance.

Sound familiar? That's where Amazon Bedrock enters the picture.

Why Serverless First?

Before getting into the AI aspects, the speakers made a quick but vital case for creating Feedbackly on a serverless architecture. This is why it made sense:

No Server Management - no patching, no provisioning, no babysitting servers
Pay Only for What You Use - no paying for idle compute between sprints
Automatic Scaling - handles bursts of feedback submissions without manual work
Faster Development - less infra, more features
Built-in Availability - AWS handles the redundancy
Focus on Business Logic - spend time on what actually matters to users

It's like renting a cab instead of buying a car. You don't have to worry about gas, insurance, or maintenance; you just get where you need to go.

Learning in Levels: The Session's Brilliant Structure

The session was set up as a series of five levels, from L1 to L5, with each level adding a new idea on top of the one before it.

L1: Bedrock Playground

The journey started with the Amazon Bedrock Chat Playground — a browser-based interface where you can experiment with multiple foundation models side by side, without writing a single line of code. It's literally a playground.

The presenters used the same feedback-classification prompt on three models at the same time: Llama 3.1 405B Instruct, Claude 3.5 Sonnet, and Command R. They wanted to see how each model reacted to the same input. The results were different for each of them. The terminology, the reasoning, the strictness, and the structure are all important. This is where it gets interesting: you need to choose a model, and the playground allows you compare them before you make a choice.

The model metrics were also interesting: Llama 3.1 had the largest latency (almost 18,000 ms), Command R was the fastest (around 1,591 ms), and Claude 3.5 Sonnet hit a sweet spot (about 4,799 ms) while giving the most structured, reasoned output.

L2: Prompt Engineering - One Word Changes Everything

AI is just as good as the directions you give it.

The presenters went over a good Prompt Template made in Amazon Bedrock's Prompt Management. It had five main parts:

Persona / Role - tell the model who it is
Action - tell it what to do
References - give it positive and negative examples to anchor its judgment
Variables - use placeholders like {{feedback}} for dynamic input
Output Format - ask for structured JSON so your application can actually parse the result

The prompt told the model to sort peer input into three groups: Reliability, Productivity, and Positive Energy. Then, it had to rank each group from 1 to 5. If there isn't enough background for a category, give it a -1.

After that, the most memorable demo of the whole session happened. They gave the identical report twice: "Person X has been productive and done the tasks as expected." What's the difference? One instruction suggested, "Be easy on the ratings." The other remarked, "Be strict with the ratings."

The outcomes were markedly distinct. One word. That's all it needed to change the AI's score. It's a strong reminder that you have to be careful when writing your prompt because it's the most important part of how your AI feature will work. Like you test your code, you should also test your prompts.

The Architecture: Lambda, RDS, and Bedrock Working Together

The basic structure of LamRAG is clean, serverless, and easy to understand:

The User sends a request
AWS Lambda receives it, validates the data, and calls both RDS and Bedrock
Amazon RDS acts as the data store, holding all sprint feedback
Amazon Bedrock takes that data, creates a query, and generates a human-readable summary

Lambda takes care of the orchestration. Bedrock is the smart part. RDS has the truth. Easy to use, works well, and is fully managed - no servers to worry about.

L3: Vector Databases and RAG

"RAG" means "Retrieval-Augmented Generation." In simple terms, you give an AI model access to your personal data instead of only what it already knows from training. This way, its replies are based on your specific situation instead of general internet knowledge.

The lecturers utilised a smart example about fruits to describe how vector databases work. Think about how you would describe an apple not just by its name, but also by its colour (1.0), sweetness (7), sourness (4), crunchiness (8), and shelf life (0.5). That list of numbers is a vector. The vector for an Orange is [0.8, 6, 8, 2, 1.0]. A vector database keeps these embeddings and uses arithmetic to locate items that are comparable to them, not keyword matching.

When you ask for "list some red-colored fruits," the database looks for vectors that are closest to the numbers that represent "red" and "fruit." That's semantic search — and it's far more powerful than a simple text search.

Feedbackly integrated feedback data stored in Amazon S3 to a Bedrock Knowledge Base. This let users to choose how to separate and index documents for quick retrieval by using configurable chunking schemes such as default, fixed-size, hierarchical, semantic, or no chunking.

L4: Agents - Smart, Conversational, and Privacy-Aware

Three months after the first Feedbackly launch, two new problems came up: Admins vs. Users access control and a problem with how feedback was being shown. Not everyone should be allowed to see what other people have said.

The solution? Bedrock Agents.

An agent is like a smart helper that can think, plan, and do things. The speakers built an agent called sls-days-2024-lamrag based on Claude 3 Haiku. The agent has the following settings:

Action Groups - A Lambda function that takes two arguments: the type of inquiry (self or others) and the email address of the person who asked for it.
Knowledge Base - linked to the Feedbackly S3 data, with the order to get data depending on the user's email address
Privacy Logic - The Lambda checks to see if you're asking about yourself or someone else and blocks access right away if it's not allowed.

The live demo was really cool. When an unauthorised email sought to get someone else's comments, the agent said, "You are not authorised to access that information." But when a user asked for their own comments, they got a long, conversational summary: "Sandeep is a very productive, dependable, and positive team member who always gets great results."

That's not only smart AI; it's also responsible AI. Privacy built into the design.

L5: Keeping the Knowledge Base in Sync

A knowledge base is only useful if it is up to date. If new feedback is sent in but the knowledge base isn't updated, the agent keeps answering questions with old information, like a librarian using last year's catalogue.

The presenters talked about this directly with L5, which kept the Bedrock Knowledge Base up to date. The knowledge base needs to re-sync every time fresh feedback is processed and uploaded to the S3 bucket (sls-days-2024-lamrag) so that the agent always has the most up-to-date information. It's a phase that is easy to forget, but it is very important for production systems.

Final Results: The AI Chat Assistant in Action

The end result was a fully functional conversational AI Chat Assistant that was built right into Feedbackly. A member of the team could type a question in plain English and get structured, data-backed answers.

For instance, asking, "What are the average ratings for this worker?" came back:

Positive Energy: 3.9/5 - Very Good
Productivity: 2.5/5 - Below Average
Reliability: 2.7/5 - Below Average
Overall: 3.1/5 - Average

Along with specific strengths (such being good at code reviews, managing time, and mentoring) and areas where they need to improve. No more empty "10/10 across the board" ratings. Instead, there will be real, detailed, AI-backed analysis based on real peer input over time.

Bonus: From Idea to App with Claude Code

The last part of the session was a bonus that honestly blew everyone away. The speakers showed how they used Claude Code, Anthropic's agentic coding tool, to construct the full-stack feedback analyser.

The workflow was deceptively simple:

Create a TASKS.md file — describe the application in plain English (tech stack, features, database setup, everything)
Tell Claude Code to refer the file and build the app
Deploy the app

The TASKS.md file listed all the parts of the stack: For the frontend, we use React, Vite, and Tailwind CSS. For the backend, we use AWS Lambda (Node.js 22). For AI, we use Claude 3.5 Sonnet via Bedrock. For the database, we use PostgreSQL on RDS. For the infrastructure, we use AWS SAM. Claude Code asked a few questions to make sure he understood, and then he made a full implementation plan. This plan included AI functions, security functions, and even privacy functions that replace colleague names with [Colleague] in queries that include more than one user.

This led to the slide that made everyone think the most: "Does this mean we don't need to learn coding anymore?"

Honestly, no, it doesn't. The skill itself is changing. It's more important than ever to know about design, what good code looks like, and how to look at AI-generated output with a critical eye. The startups that the speakers talked about - Cursor, Midjourney, Lovable, and Eleven Labs - were all started by small groups of people that employed AI to help them work faster, not to replace them.

Key Takeaways

This was one of the most useful AI sessions I've ever been to. This is what I'm taking with me:

Prompt engineering is a real, learnable skill. The word "lenient" vs. "strict" makes all the difference. Check your prompts the same way you check your code.
RAG makes AI relevant to your world. Foundation models are not specific. RAG makes them fit your data, your people, and your situation.
Agents add intelligence and access control together. Bedrock Agents can think about who is enquiring before they decide what to respond.
Serverless + Bedrock is a genuinely practical stack. You can send AI features that are ready for production without having to manage a single server.
AI amplifies builders, it doesn't replace them. The actual expertise of this time is knowing what to develop and how to direct the AI.

The Amazon Bedrock Chat Playground is the best place to start if you want to attempt any of this yourself. You don't need any code; simply open your browser and start playing around.

Conclusion

The LamRAG session at AWS Student Community Day Tirupati reminded me that the finest tech speeches don't only teach you ideas; they also show you a real problem, a real solution, and a genuine way to move forward. In short, here's the broad picture:

Generative AI on AWS is approachable - The Bedrock Playground enables anyone start experimenting without having to write any code.
The journey from a simple prompt to a full RAG-powered agent is incremental - You don't have to develop it entirely at once. Start with a small part and add more intelligence as you go.
Privacy and access control aren't an afterthought - Bedrock Agents let you change how the AI reacts right away.
AI-assisted development tools like Claude Code are changing the speed of building - faster than ever from idea to app in use
The best time to start learning Amazon Bedrock is right now - The tools are well-developed, the documentation is good, and the community is developing quickly.

About the Author

🔗 Connect with me on LinkedIn

References

Event: AWS Student Community Day Tirupati

Topic: LamRAG: AI-Powered Feedback Analysis Using Amazon Bedrock

Date: November 01, 2025

Also Published On

AWS Builder Center

Hashnode

Building a Private GPT with AWS Bedrock: A Deep Dive

N Chandra Prakash Reddy — Tue, 03 Feb 2026 14:18:57 +0000

How It All Started

On November 1, 2025, I attended the AWS Student Community Day in Tirupati, and honestly, one session significantly changed my perspective on developing AI applications. Amin Ali, a Software Engineer at Target Australia and an AWS-certified Solutions Architect, delivered a presentation titled "Generative AI in Action." The session concentrated on how to build your own private GPT using AWS Bedrock.

The reality is that we've all interacted with ChatGPT, haven't we? Have you ever considered how you might develop something similar that operates with your organization's internal files, retains all the information in your own cloud, and avoids transmitting any data to external APIs? This session focused on that topic, and I'm thrilled to share my insights.

Why Build Your Own Private GPT?

You might wonder why you can't just use tools like ChatGPT or Claude directly. That's a good question, and I can explain with an example.

Data Control: Your Information, Your Rules

Imagine you run a hospital and need an AI to help doctors find patient information fast. Would you want to give private medical data to a public AI service? Probably not. By creating your own private GPT, you control your data within your AWS system, so you don't need outside services. Your private information stays safe in your Virtual Private Cloud. You'll also have full records of everything that happens, which helps follow strict rules, and no data is ever shared with anyone else.

Customization: Make It Yours

Public AI models usually learn from information found online. But what if you need answers that are only about your company? A private GPT can give answers based on your own files and knowledge, allowing it to sound like your company. It's like a customer helper who knows everything about your products. It can also connect easily with your company's internal systems, such as employee records, product lists, or instruction guides.

Understanding Generative AI

First, let's be sure we all know what generative AI is. Amin explained it well with a three-step process.

Input Processing: You tell the AI what to do, like giving it words, pictures, or code. Big AI programs like GPT, Claude, or Llama, which have learned from tons of information, handle these instructions.

Foundation Models: Regular machine learning finds patterns in data to guess things, like if an email is spam. But large language models create brand new things from nothing. That's why they are called "generative."

Creative Output: The AI makes new words, pictures, code, and other things that didn't exist before. It's like having a helpful assistant that can create, write code, and solve problems.

Why Generative AI Is Exploding Right Now

Now is a good time to learn about this new technology. Generative AI is popular for these reasons:

The AI Revolution: Many advanced AI models like ChatGPT, Claude, Mistral, and Llama 3 are becoming more common, and they are significantly altering how we work and create.

Enterprise Adoption: AWS is assisting companies in building reliable and secure generative AI systems that can manage a lot of users. Businesses don't have to build everything from scratch anymore.

Industry Impact: Smart automation is transforming industries like healthcare, shopping, schools, and banking. Think about doctors available 24/7 or educational tools that adjust to each student's needs.

Global Scale: All kinds of companies, from small new businesses to large established ones, are using generative AI a lot, and this trend is growing quickly.

Real-World Use Cases

Let's discuss practical applications. This isn't just theory these are real scenarios where private GPTs genuinely make a difference.

Customer Support: AI chatbots provide 24/7 personalized assistance that is as effective as a human helper. Imagine a customer checking your return policy late at night and getting a fast, accurate response that aligns with your actual rules.

Report Generation: Automatically generating business reports, summaries, and documents. Rather than spending a lot of time compiling quarterly reports, the AI collects the data and creates well-organized documents on its own.

Document Q&A: An internal knowledge base that enables quick answers based on company documents. Picture a new employee asking, "What is our vacation policy?" and the AI quickly finding the answer in the employee handbook.

Developer Assistants: Generating code, fixing errors, and writing explanations to speed up the development process. Programmers can ask, "How do I connect to our database?" and receive code examples that work with your system.

The Live Demo: MBU Virtual Assistant

This is where things became truly intriguing. Amin demonstrated an AI assistant he developed for Mohan Babu University.

The interface was smooth simple chat design built with React that appeared both professional and user-friendly. When a question such as "What programs does MBU offer?" was asked, the system would process the query through the chat, scan the uploaded university documents using vector similarity, generate a contextually appropriate response with AWS Bedrock, and deliver the answer in real time.

The demo showcased the chat interface with live-streaming responses and Bedrock integration performing real-time searches on the knowledge base using vector search examples. Powered by Amazon Bedrock, the responses flowed smoothly, creating the impression of conversing with an intelligent assistant.

How the Architecture Works

Let me guide you through the system's structure. Don't worry I'll keep it simple and easy to understand.

The User Journey

Here's the journey of a user's query through the system:

User Interaction: You enter a question using the React-based web interface.
API Gateway: Your request reaches AWS API Gateway, which serves as the main entry point (REST endpoint).
Lambda Function: API Gateway forwards the request to an AWS Lambda function imagine this as the central component that manages all operations.
Bedrock Integration: Lambda calls AWS Bedrock's Knowledge Base to perform a vector search and uses the Large Language Model to create the response.
Response Delivery: The answer then travels back through the same route and appears on your screen.

Architecture Components

Amin broke down the architecture into clear layers using a complete serverless setup:

This serverless configuration means you don't need to manage any servers. AWS handles scaling and security automatically.

Setting Up the Knowledge Base

This is where the real action takes place. How can you train the AI on your particular documents?

Data Preparation Pipeline

The process follows three key steps:

Upload PDFs: Your documents (manuals, policies, guides) are uploaded to an S3 source bucket.
Extract & Chunk: The system pulls out the text and divides it into smart segments for the best vector embeddings.
Generate Embeddings: AWS Bedrock generates vector embeddings that are saved in an S3 vector bucket for similarity searches.

Vector Integration

Does this ring a bell? Imagine vector embeddings as a library catalog. Instead of just matching exact words, vectors understand the meaning behind them.

The Bedrock Knowledge Base connects to an S3 vector bucket with 2024-dimensional embeddings. When you ask a question, the system finds relevant sections of documents using cosine similarity, searches for matching info in real time, and provides answers based on the most important details.

The setup involves connecting to an S3 vector bucket, automatically updating indexes, and using Bedrock directly – all these components work together seamlessly.

Query Handling Flow

Let's trace a question through the whole system.

Step 1 - User Query: You type a question into the React chat interface, just like you would in any messaging app.

Step 2 - Vector Search: Lambda checks the Bedrock Knowledge Base for relevant document sections. The system finds the 3-5 most relevant pieces of information from your document collection.

Step 3 - Context Response: The Large Language Model creates a response based on the retrieved context and sends it back to the user interface. The answer is based on your actual documents, not general internet information.

Best Practices and Optimization

Building the system is one thing, but making it run well is different. Amin shared important tips for improvement.

Prompt Engineering

Writing good prompts helps get better and more useful answers.

Use clear, specific instructions. Instead of saying "tell me about products," say "list the top 3 features of Product X based on our product documentation."
Add context and examples to your prompts.
Improve your prompts based on the responses you get, prompt engineering is a process that requires adjustments.

Performance Optimization

Speed and cost are important in production

Allow real-time responses so users can see answers as they come in, which improves the user experience.
Use tokens wisely, keep prompts short since tokens cost money.
Set up caching for frequently asked questions.

Key Takeaways

After witnessing this in action, three key points caught my attention:

AWS Bedrock: Secure and Flexible: AWS Bedrock provides a secure and flexible foundation for developing private AI solutions. You receive strong security features without needing to build everything yourself.

Production Ready: Generative AI is now ready for business use and has demonstrated real results that justify the investment. This is no longer just experimentation – companies are implementing these systems in their daily operations.

Easy Implementation: Building a private GPT is simpler than ever with managed services. You don't need to be a machine learning expert to get started.

Conclusion

After leaving that session, I realized that AI isn't just about using ChatGPT. It's about building smart systems tailored to your specific needs while keeping your information secure.

The AWS Student Community Day in Tirupati had many great sessions, but this one impressed me because it demonstrated how to turn an idea into a working demo. Whether you're making a company assistant, a customer support bot, or a documentation helper, the architecture Amin presented provides a solid foundation.

In summary, if you're thinking about adding AI features to your apps, AWS Bedrock makes it simpler than you might expect. With managed services, strong security, and flexibility, you can focus on solving your specific problem instead of building the entire AI setup yourself.

I hope this explanation helps you understand how private GPTs work and inspires you to create your own. The future of AI will be private, secure, and tailored to your personal needs.

About the Author

_As an AWS Community Builder, I enjoy sharing the things I've learned through my own experiences and events, and I like to help others on their path. If you found this helpful or have any questions, don't hesitate to get in touch! 🚀

🔗 Connect with me on LinkedIn_

References

Event: AWS Student Community Day Tirupati

Topic: How AWS Lambda and Fargate Change the Way We Build Apps

Date: November 01, 2025

Also Published On

AWS Builder Center

Hashnode