DEV Community: Vuong Bach Doan

[VN] Hướng dẫn Cài đặt CloudWatch Agent để Giám sát Lưu lượng Lightsail (Ubuntu OS)

Vuong Bach Doan — Thu, 10 Jul 2025 06:44:20 +0000

Tóm tắt

Thiết lập cảnh báo cho lưu lượng truyền tải mỗi tháng khi vượt quá 50% hoặc 80% của 4TB (giới hạn miễn phí của gói Lightsail 4GB):

Bước 1: Lấy metric Network In/Out từ CloudWatch.
Bước 2: Tạo cron job tính tổng lưu lượng.
Bước 3: Tạo cảnh báo (alarm) khi vượt ngưỡng.

Trước khi bắt đầu, cài AWS CLI

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

1. Khởi tạo IAM User

Truy cập AWS IAM Console.
Chọn Users > Create user.
Nhập tên: lightsail-cloudwatch-agent.
Chọn Attach policies directly và gán policy CloudWatchAgentServerPolicy.
Nhấn Next, thêm tag (tùy chọn), và chọn Create user.
Sau khi tạo, chọn user > Create access key.
Chọn CLI use case, tải file .csv chứa Access key và Secret.

2. Cài đặt và Cấu hình CloudWatch Agent

Bước 1: Cài đặt CloudWatch Agent

sudo apt update
sudo apt install -y wget unzip
wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
sudo dpkg -i amazon-cloudwatch-agent.deb

Bước 2: Thiết lập credential profile

aws configure --profile AmazonCloudWatchAgent

Nhập Access Key và Secret từ bước trước.
Region: ap-southeast-1
Output format: để trống hoặc json.

3. Cấu hình CloudWatch Agent

Bước 1: Tạo file config metric

sudo nano /opt/aws/amazon-cloudwatch-agent/bin/config.json

Dán nội dung sau:

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "root",
    "region": "ap-southeast-1"
  },
  "metrics": {
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "net": {
        "measurement": [
          "bytes_recv",
          "bytes_sent"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "*"
        ],
        "total": true
      }
    }
  }
}

Bước 2: Thiết lập profile sử dụng

sudo nano /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml

Dán nội dung sau:

[credentials]
shared_credential_profile = "AmazonCloudWatchAgent"

Lưu lại bằng tổ hợp Ctrl + O, Enter, sau đó Ctrl + X.

Bước 3: Khởi động Agent

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

Kiểm tra trạng thái:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a status

4. Kiểm tra Metric Trên CloudWatch

Vào CloudWatch Console.
Chọn Metrics > All metrics > Browse.
Kiểm tra tại Custom namespaces > CWAgent.

Để tìm Instance ID và interface:

aws cloudwatch list-metrics \
  --namespace "CWAgent" \
  --metric-name "net_bytes_recv" \
  --region ap-southeast-1 \
  --output json

5. Cron Job Tính Tổng Lưu Lượng Mỗi Tháng

Bước 1: Tạo script

nano ~/network_monthly_usage.sh

Dán nội dung sau (chỉnh INSTANCE_ID và INTERFACE_NAME):

#!/bin/bash

export AWS_PROFILE=AmazonCloudWatchAgent

INSTANCE_ID="i-07b283c143999f4d0"
INTERFACE_NAME="eth0"
REGION="ap-southeast-1"

START_DATE="$(date -u +%Y-%m-01T00:00:00Z)"
END_DATE="$(date -u +%Y-%m-%dT%H:%M:%SZ)"

RECV=$(aws cloudwatch get-metric-data --metric-data-queries "[{\"Id\":\"recv\",\"MetricStat\":{\"Metric\":{\"Namespace\":\"CWAgent\",\"MetricName\":\"net_bytes_recv\",\"Dimensions\":[{\"Name\":\"InstanceId\",\"Value\":\"$INSTANCE_ID\"},{\"Name\":\"interface\",\"Value\":\"$INTERFACE_NAME\"}]} ,\"Period\":86400,\"Stat\":\"Sum\"},\"ReturnData\":true}]" --start-time "$START_DATE" --end-time "$END_DATE" --region "$REGION" --query 'MetricDataResults[0].Values' --output text | awk '{sum+=$1} END {print sum+0}')

SENT=$(aws cloudwatch get-metric-data --metric-data-queries "[{\"Id\":\"sent\",\"MetricStat\":{\"Metric\":{\"Namespace\":\"CWAgent\",\"MetricName\":\"net_bytes_sent\",\"Dimensions\":[{\"Name\":\"InstanceId\",\"Value\":\"$INSTANCE_ID\"},{\"Name\":\"interface\",\"Value\":\"$INTERFACE_NAME\"}]} ,\"Period\":86400,\"Stat\":\"Sum\"},\"ReturnData\":true}]" --start-time "$START_DATE" --end-time "$END_DATE" --region "$REGION" --query 'MetricDataResults[0].Values' --output text | awk '{sum+=$1} END {print sum+0}')

TOTAL_BYTES=$(echo "$RECV + $SENT" | bc)
TOTAL_GB=$(echo "scale=3; $TOTAL_BYTES / 1024 / 1024 / 1024" | bc)

aws cloudwatch put-metric-data \
  --namespace "Custom/MonthlyNetworkTraffic" \
  --metric-name "TotalNetworkTrafficBytes" \
  --value "$TOTAL_BYTES" \
  --unit Bytes \
  --region "$REGION"

echo "Total monthly traffic so far for $INSTANCE_ID ($INTERFACE_NAME): ${TOTAL_GB} GiB"
echo "Metric pushed to CloudWatch: Custom/MonthlyNetworkTraffic → TotalNetworkTrafficBytes"

Cấp quyền thực thi:

chmod +x ~/network_monthly_usage.sh

Bước 2: Tạo cron job

crontab -e

Thêm dòng sau (chạy mỗi phút):

* * * * * /bin/bash /home/ubuntu/network_monthly_usage.sh >> /home/ubuntu/network_usage.log 2>&1

Kiểm tra log:

tail -n 50 /home/ubuntu/network_usage.log

6. Tạo Alarm Cảnh Báo

Truy cập CloudWatch Console.
Vào Alarms > All alarms > Create alarm > Select metric.
Chọn namespace: Custom/MonthlyNetworkTraffic.
Chọn metric: TotalNetworkTrafficBytes.
Thiết lập threshold:

-   Statistic: Average

-   Period: 1 minute

-   Threshold type: Static

-   Condition: Greater than or equal to `2199023255552` (Bytes = 50% của 4TB)

Chọn gửi cảnh báo qua email với SNS:

-   Chọn Create new topic

-   Đặt tên và nhập email nhận cảnh báo

-   Xác nhận email khi nhận được

Đặt tên cho alarm, xác nhận và tạo.

7. Tạo Alarm thứ hai (80%)

Lặp lại các bước trên, thay giá trị ngưỡng thành 3518437208883 Bytes (~80% của 4TB).

✅ Kết luận

Khi lưu lượng vượt ngưỡng 50% hoặc 80%/tháng, hệ thống sẽ gửi cảnh báo email.

Building a Geometry Dash-Style Game with Amazon Q and AI Tools

Vuong Bach Doan — Tue, 27 May 2025 17:19:04 +0000

Vuong Bach Doan for AWS Community Builders

May 27 '25

I build a game within 15 minutes with Amazon Q CLI and AI Tools

#aws #awschallenge #amazonqcli

Comments 1

3 min read

I build a game within 15 minutes with Amazon Q CLI and AI Tools

Vuong Bach Doan — Tue, 27 May 2025 17:18:09 +0000

Building a Geometry Dash-Style Game with Amazon Q and AI Tools

Overview

This guide walks through the process of creating a simple Geometry Dash-inspired game using Amazon Q, Bedrock, and various creative tools. The main steps include:

Drafting a game idea and refining it into a structured prompt using Bedrock (Claude 3 Haiku)
Generating the initial game project with Amazon Q CLI
Enhancing visuals and audio using Figma, Stable Audio & Claude 3 Sonet, and fonts from itch.io
Writing a helpful README.md with assistance from Amazon Q CLI

Prerequisites

Amazon Q CLI installed and configured
Python installed on your system

Step 1: Create a High-Quality Prompt with Bedrock

To kickstart the project, I drafted my initial game idea and used Amazon Bedrock (Claude 3 Haiku) to help refine it into a well-structured prompt for Amazon Q.

My initial idea:

I have an idea of using Amazon Q to create a game with Pygame, the game style is like Geometry Dash. The player presses SPACE to jump, R to restart the game. Include an "x2" coin that boosts speed and doubles coin collection. Provide a well-structured prompt for Amazon Q to generate the game project.

Claude's refined prompt:

Create a 2D side-scrolling platform game inspired by Geometry Dash using Pygame with the following features and requirements:

## Game Mechanics:
- Auto-forward movement
- Jump with SPACE
- Restart with R
- Coins to collect
- "x2" power-up that doubles speed and coin value

## Technical Implementation:
- Object-oriented structure
- Classes for game manager, player, obstacles, coins, power-ups, UI

## Visuals and Controls:
- Simple geometric character
- Spikes and obstacles
- Parallax scrolling background
- Score counter and game over screen

## Additional Features:
- Sound effects and music
- Particle effects
- Progressive difficulty
- High score system

Please use modular code files and implement proper game loops, collision detection, and resource management.

Run q chat, paste the prompt, and Amazon Q will generate a basic folder structure like the one below:

Step 2: Enhance the Game with Custom Assets

🎵 Generate a Custom Theme Song with Stable Audio

I used Claude 3 Sonet via Bedrock to generate a prompt for a custom game theme:

## Geometry Dash-Style Electronic Theme Song
- Tempo: 160 BPM
- Key: F minor
- Loop duration: 30s seamless loop
- Instruments: bright saw synth, digital tom rolls, square wave arpeggios, side-chained kick and snare, reverse cymbals

Paste this into Stable Audio to generate a high-energy game soundtrack.

You can download my generated theme song here: Cloud Dash Song

🧍‍♂️ Create Custom Characters with Figma

Use Figma to draw and export a unique character sprite for your game.

🔤 Add Pixel-Style Fonts

You can find retro and pixel-style fonts on itch.io to enhance the game's visual appeal.

🛠️ Fix Bugs and Add Features with Amazon Q

Use q chat to ask Amazon Q for help with fixing bugs or extending game logic. It can assist in:

Debugging game logic
Adding new mechanics
Refactoring code

Also, ask Amazon Q to write your README.md file for better project documentation.

DEMO

All source code will be on my GitHub repo: Cloud Dash

From Capstone to Cloud: Deploying a Steam-like Platform for Students with AWS 🚀

Vuong Bach Doan — Wed, 21 May 2025 08:27:36 +0000

Vuong Bach Doan for AWS Community Builders

May 21 '25

Building GameHub Platform: A Steam-like Ecosystem for Students with AWS Cloud

#aws #devops #kubernetes #microservices

Comments 2

5 min read

Building GameHub Platform: A Steam-like Ecosystem for Students with AWS Cloud

Vuong Bach Doan — Wed, 21 May 2025 08:18:26 +0000

1. Introduction

In this blog post, I’ll share my journey creating GameHub Platform, a student-focused gaming ecosystem inspired by Steam, built using a variety of AWS services.

For security reasons, I won’t reveal sensitive implementation details. Instead, I’ll provide clear architectural insights and technical highlights so you can envision and build a similar system on your own.

2. Problem Statement

At FPT University, two majors naturally complement each other: graphic design and game development. Students in graphic design create stunning 2D and 3D assets, while game developers craft engaging games. Yet, despite their synergy, these groups lacked a shared platform to collaborate, showcase, and monetize their work.

Here’s the gap GameHub Platform aims to fill:

Graphic design students can upload and display their game assets like images, models, and animations.
Game development students browse, download, and use these assets to build their games.
Finished games get published and sold right on the platform.

The result? A vibrant ecosystem that promotes learning, creativity, and entrepreneurship among students - a mini Steam tailored for campus talents.

3. Architecture Overview

Here’s a high-level look at the GameHub Platform’s architecture, organized into three layers:

3.1 CI/CD Pipeline

Source code for frontend and backend lives in GitHub.
On each commit, GitHub Actions builds Docker images and pushes them to Amazon ECR (Elastic Container Registry).
Automated deployments trigger updates to development, staging, and production environments.

3.2 Cloud Environment

Backend services run on Amazon EKS (Elastic Kubernetes Service), handling traffic via a Load Balancer.
Persistent data is stored in a combination of Amazon RDS for relational data, MongoDB Atlas for flexible metadata, and Amazon S3 for asset storage.
Frontend UI is served via AWS Amplify and cached globally with CloudFront CDN.
Security is enforced using AWS WAF, Security Groups, and network policies.

3.3 Feature Services

Additional services enrich the platform:

Payment Gateway integrates Stripe/PayOS for smooth transactions.
Student ID Verification uses Amazon Textract for automated card scanning.
Virus Scanning ensures uploaded files are safe using the VirusTotal API.
Real-time Notification Service alerts users of important events.

4. Tech Stack & Key Services Used

Choosing the right tech stack was critical to ensuring the platform could scale, stay secure, and remain cost-effective. Here’s a snapshot of the core tools and why they were chosen:

Service / Tool	Category	Why It Was Chosen
Next.js + TailwindCSS	Front-End Framework	Fast development with modern UI utilities and great SEO support.
AWS Amplify + CloudFront	Front-End Hosting & CDN	Low-latency global distribution for frontend assets.
.NET Core (Dockerized)	Back-End API	Robust, cross-platform, high-performance microservices.
Amazon ECR	Container Registry	Seamless integration with AWS services and CI/CD pipelines.
GitHub Actions	CI/CD Pipeline	Native GitHub integration, flexible YAML workflows.
Amazon EKS (Kubernetes)	Container Orchestration	Managed Kubernetes with scaling, high availability, and strong AWS integration.
RabbitMQ	Message Broker	Decouples services and handles asynchronous tasks efficiently.
Amazon RDS (MySQL)	Relational Database	Reliable, managed SQL database with automated backups.
Amazon ElastiCache (Redis)	Caching Layer	Speeds up frequent queries, reduces load on databases.
Amazon OpenSearch	Log Aggregation & Search	Centralizes logs for monitoring and troubleshooting.
Amazon S3	Object Storage	Durable storage for game assets, screenshots, and other user uploads.
AWS WAF + Security Groups	Security	Protects infrastructure from malicious traffic and controls access.
Amazon Route 53 + ACM	DNS & SSL	Custom domain routing and free TLS certificates.
Amazon CloudWatch	Monitoring & Logging	Provides metrics, alarms, and insights for operational health.
AWS Lambda	Event-Driven Functions	Lightweight, on-demand compute for scanning, notifications, and integration.
VirusTotal API	Malware Detection	Automatically scans uploaded files to detect threats.
Amazon Textract	Document Extraction	Extracts and verifies student ID card data automatically.
Stripe / PayOS	Payment Processing	Secure and simple API for processing payments.
MongoDB Atlas	NoSQL Data Store	Flexible metadata storage for assets and user data.

A thoughtfully chosen tech stack forms the foundation for a reliable, scalable, and maintainable platform.

5. Deep Dive: Back-End Services on EKS with gRPC

To handle the platform’s complexity, the backend is split into microservices running in a Kubernetes cluster on Amazon EKS.

5.1 Microservices Breakdown

Key services include:

Main Service: Manages games, assets, and user interactions.
Payment Service: Handles all transaction-related logic.
Role Management: Manages user permissions and access levels.
Notification Service: Sends emails, in-app alerts, and push notifications.

Each service runs in its own Kubernetes pod, enabling independent scaling and easier updates.

5.2 Why Kubernetes (EKS)?

EKS was selected because:

It offers automatic scaling and high availability.
Deep integration with AWS services (IAM, CloudWatch, VPC).
Simplifies deployments using standard Kubernetes tooling and GitOps workflows.

This allows the platform to efficiently adapt to varying traffic from student uploads, downloads, and transactions.

5.3 gRPC for Inter-Service Communication

Instead of REST APIs, services communicate via gRPC using protocol buffers:

Feature	Benefit
Performance	Compact binary format for faster serialization.
Low Latency	Efficient use of HTTP/2 improves speed.
Strong Typing	Protobuf contracts reduce errors.
Bi-directional	Supports streaming data and advanced RPCs.

This ensures fast, reliable, and maintainable communication inside the Kubernetes cluster.

5.4 Security

gRPC traffic is confined to the private VPC network.
Kubernetes Network Policies restrict service-to-service access.
Public-facing endpoints are secured with authentication and AWS WAF rules.
IAM and Kubernetes RBAC policies enforce least privilege access.

6. Feature Services

Beyond core gameplay and asset management, GameHub Platform integrates specialized services to boost security and usability:

6.1 VirusTotal for Malware Scanning

Before any game or asset is published, it is scanned via the VirusTotal API to detect viruses or malware. This protects the community from harmful content and builds trust in the platform.

6.2 Amazon Textract for Student ID Verification

Users upload photos of their student ID cards. Amazon Textract extracts textual data, which is then cross-checked against the university’s database to verify enrollment status. This helps gatekeep student-only features.

6.3 Payment Gateway Integration

Payments are handled via Stripe or PayOS, offering students an easy and secure way to buy and sell games and assets.

6.4 Notification Service

Real-time alerts via email and in-app notifications keep users informed about new assets, transactions, and platform updates.

6.5 Storage and Asset Management

Amazon S3 provides scalable, secure storage for all uploaded content, backed by lifecycle policies and encryption.

7. Conclusion & Lessons Learned

Building GameHub Platform was a rewarding journey that taught me a lot about cloud architecture, container orchestration, and integrating multiple AWS services to build a real-world application.

Key takeaways:

Modular microservices running on Kubernetes offer flexibility and scalability.
Using gRPC improves performance and maintainability for internal APIs.
Cloud-native services like Amazon Textract and VirusTotal APIs add critical security layers.
Automating deployments with CI/CD pipelines accelerates delivery and reduces errors.

If you’re inspired to build your own cloud-native platform or want to learn more about AWS, Kubernetes, and modern microservices design, feel free to reach out. I’m happy to share insights or collaborate!

Let’s keep innovating and building great things together!

You can contact me at:
📧 vbachdoan@gmail.com
🔗 LinkedIn Profile: vuongbd2007

Thank you for reading and happy coding! 🚀

Terraform Automation tool powered by Amazon Q Developer

Vuong Bach Doan — Mon, 05 May 2025 10:21:09 +0000

This is a submission for the Amazon Q Developer "Quack The Code" Challenge: Crushing the Command Line

What I Built

I built a CLI-based automation tool called Terraform Automation Tool that simplifies and streamlines the process of managing and bootstrapping AWS infrastructure lab environments using Terraform. The tool allows developers and educators to:

Choose from a list of pre-configured AWS labs (e.g., 3-tier architecture, static websites, RDS setup).
Automatically generate, configure, and deploy the necessary Terraform code.
Customize lab scaffolding interactively using a cross-platform terminal interface.

This solves the problem of manual setup and error-prone provisioning of AWS labs, especially useful for students, trainers, and workshop facilitators who want to spin up consistent and reproducible environments quickly.

Demo

Here's a quick demo of the tool in action:

The CLI interface is intuitive and works across macOS, Linux, and Windows.

Code Repository

Explore the full project here:
🔗 GitHub – vuongbachdoan/terraform-automation-bot

How I Used Amazon Q Developer

Amazon Q Developer was instrumental in shaping both the design and functionality of this CLI-based tool. It supported me in building a smart, task-driven menu with the following interactive options:

? What do you want to do?
>   ✨ Generate Terraform Folder Structure
    ✨ Optimize Terraform Source Code
    ✨ Check for Security Issues
    ✨ Deploy Terraform Resources
    ✨ Destroy Terraform Resources

Here’s how Amazon Q Developer contributed throughout the development process:

🛠️ Generated Terraform scaffolding logic for the “Generate Terraform Folder Structure” option, ensuring clean, modular layouts that follow best practices.
🚀 Recommended optimizations for Terraform source code by highlighting unused variables, redundant resources, and performance tweaks.
🔐 Guided me in integrating security checks, helping build logic to detect exposed credentials, misconfigured IAM policies, and open ports.
📦 Assisted with deployment and teardown workflows by suggesting reliable patterns for terraform apply and terraform destroy, including state locking and error handling.

Amazon Q served as a real-time AI collaborator that accelerated the build process, reduced trial-and-error, and made the CLI smarter and safer for any user working with AWS infrastructure.

⚠️ By submitting this entry, I agree to receive communications from AWS regarding products, services, events, and special offers. I can unsubscribe at any time. My information will be handled in accordance with AWS's Privacy Policy. Additionally, my submission and project may be publicly featured on AWS's social media channels or related promotional materials.

Connecting to a Private RDS Instance via an EC2 Proxy

Vuong Bach Doan — Wed, 09 Apr 2025 03:59:45 +0000

Context

To enhance security, RDS instances are typically placed in a private subnet within a VPC, making them inaccessible from outside the VPC.

In dev/test environment, this can pose a challenge when you need to connect to the database from your local machine to test your application.

About this solution

This solution provides a way to set up a proxy that forwards traffic from your client to the private RDS instance using an EC2 proxy instance.

You will setup an EC2 instance, at public subnet that it can receive request from developer.
The EC2 instance will forward traffic to RDS instance at private subnet

Architecture Overview

Setup

1. Create EC2 Instance

Create an EC2 instance with following setting:

AMI: Amazon Linux 2 Kernel 5.10
Instance type: t2.micro
Key pair: process without a key pair
Network
- VPC: choose the VPC that same with VPC you are using with RDS instance
- Subnet: choose public subnet
- Security Group: allow all inbound traffic

Then click Launch instance to create proxy server.

2. Collect RDS instance information

To forward traffic to RDS instance, you need to know:

DB Endpoint: example.XXXXXXXXXXXX.ap-southeast-1.rds.amazonaws.com
DB Port: 3306 or 5432,...

3. Config Firewall

For proxy server security group:
- Inbound: allow Anywhere-Ipv4 (MYSQL/Aurora, PostgreSQL,... depend on your database engine) on port (3306, 5432,... depend on your database port)
- Outbound: security group of RDS instance.
For security group of database instance:
- Inbound: add additional rule to allow traffic from security group of proxy server. Don't remove any existing rule.
- Outbound: add additional rule to allow traffic to security group of proxy server. Don't remove any existing rule.

4. Setup Proxy

SSH to the EC2 instance and run following commands:

yum install haproxy -y

Replace content of /etc/haproxy/haproxy.cfg with following values:

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /var/run/haproxy.sock mode 660 level admin
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    option  dontlognull
    option  httplog
    timeout connect 5000ms
    timeout client  50000ms
    timeout server  50000ms

frontend mysql_front
    bind *:<YOUR_DB_PORT>
    mode tcp
    default_backend mysql_back

backend mysql_back
    mode tcp
    server db_server <YOUR_DB_ENDPOINT>:<YOUR_DB_PORT> check

<YOUR_DB_ENDPOINT>: RDS Endpoint
<YOUR_DB_PORT>: Database port

5. Test Connection

Now you can connect to RDS instance in private subnet by replace the RDS Endpoint with Public IP of EC2 proxy server.

6. Cleanup Resources

Delete EC2 proxy server if you no longer need it to reduce cost.

Disclaimer

This solution is prefer using for dev/test environment. For production workload, be careful when manage firewall to ensure secure connection.

🚀 BE HAPPY ON CLOUD 🚀

AWS App Studio

Vuong Bach Doan — Thu, 14 Nov 2024 03:43:10 +0000

Overview

App Studio is a generative AI-powered service that uses natural language to build enterprise-grade applications, empowering a new set of builders to create applications in minutes. With App Studio, technical professionals without deep software development skills, such as IT project managers, data engineers, and enterprise architects, can quickly develop business applications tailored to their organization's needs.

Benefits and features

Accelerate application development with a generative AI–powered assistant

Simply describe the application you want to build using natural language, App Studio will deliver an application with a multi-page user interface (UI), a data model, and custom business logic to your needs. You can then ask questions about how to modify your application, and App Studio will guide you with providing detailed answers on how to make changes using the point-and-click interface.

Deploy highly scalable applications without operational overhead

With App Studio, you get a highly secure, scalable, and performant application and you do not have to think about the underlying code or infrastructure—App Studio handles all deployment, operations, and maintenance, freeing technical professionals to focus on innovating rather than application management.

Enhance security and governance that delights IT

App Studio provides granular access control policies over your data, users, and applications, also gives IT teams visibility into every application, enabling them to track usage and maintain compliance with company policies.

Save up to 80% versus comparable offerings

App Studio upends the pricing models of existing low-code tools that charge expensive per-end user licenses. With App Studio, it’s free to build an application, and you only pay when end users interact with published applications.

Use cases

Streamline claims processing

With App Studio, you can build an application that automates the manual and repetitive tasks of claims processing in minutes. The application can deal with large volumes of claims in custom processing workflows, such as data entry, file uploads, and status tracking.

Manage inventory and equipment

Build an application that gives you a centralized view of inventory levels, equipment assignments, and reordering needs. You can connect your application to data stores to track parts compatibility information and record inventory levels. The application can automate notifications when compatibility complaints and returns reach specified levels, reducing operational costs through efficient inventory management.

Easily approve projects

Build an application for project managers that centralizes the project intake, review, and approval processes for your organization. The application can increase visibility into project status and approver information as well as automate notifications and integrate with existing tools for contract management and project kickoffs.

[From Whizlab] AWS Cheatsheet

Vuong Bach Doan — Tue, 05 Nov 2024 04:37:07 +0000

Generative AI Call Center

Vuong Bach Doan — Thu, 24 Oct 2024 06:31:08 +0000

First Cloud Journey Cloud Day - Generative AI Call Center

This guide provides step-by-step instructions to create a call center solution that integrates Amazon Bedrock as an AI assistant and uses OpenAI's Whisper model for real-time voice-to-text transcription. The implementation will utilize Amazon SageMaker to deploy Whisper and Amazon Bedrock to enhance the interaction with AI-driven responses.

Prerequisites
Architecture Overview
Setup Instructions
- Step 1: Create Whisper Endpoint
- Step 2: Deploy Lambda Function
- Step 3: Set Up Amazon Connect Flow
Congratulations

Prerequisites

Download resource at here

or using

git clone https://github.com/vuongbachdoan/fcj-cd-demo.git

Before getting started, ensure you have the following:

An AWS account with access to:
- Amazon Connect
- Kinesis Video Stream
- Bedrock
- SQS
- Lambda
- SageMaker
- DynamoDB
Basic knowledge of Python, Jupyter Notebook, and familiarity with AWS services.
AWS CLI and SageMaker Studio set up on your machine.

Architecture Overview

Setup Instructions

Step 1: Create Whisper Endpoint

About Whisper

The Whisper endpoint is a powerful tool for speech recognition, offering extensive capabilities for transcription and understanding spoken language across various applications. It leverages advanced machine learning techniques to deliver accurate and efficient transcriptions.

How it Works

The Jupyter notebook deploys a SageMaker endpoint with a custom inference script similar to this example in the SageMaker SDK documentation. The components required to deploy a pre-trained model to an endpoint in SageMaker are:

A serialized model artifact (tar file) in Amazon S3.
The code and requirements that run inference.

These components are packaged into a SageMaker endpoint, which serves the serialized model with custom code behind it as an API. See the architecture below for a visual description.

Setup SageMaker Notebook

Navigate to the SageMaker dashboard.
Go to Application and IDEs > Notebooks in the left sidebar.
Click Create notebook instance and complete the following configurations:
- For Notebook instance name, enter a unique name.
- For Notebook instance type, choose ml.m5.2xlarge.
- Expand Additional configuration and set the Volume size in GB to 10GB.
- Leave other fields as default.
Wait until the status of the notebook instance changes to In service before using the instance.
Open JupyterLab to run your notebook.

Run Notebook

Navigate to the /whisper folder in your current repository.
Copy all files and folders, upload them to the current notebook, and run the notebook.
It is recommended to run every cell one by one.
The folder structure will look like this:

You will have an endpoint named whisper-endpoint under Inference > Endpoints.

Step 2: Deploy Lambda Function

Navigate to /amazon-connect in your current repository.
Run the following command on your local computer:
```
aws configure
```

Then enter your AWS Access Key ID, AWS Secret Access Key, and Default region name (us-west-2).

Clone the repository:

git clone https://github.com/vuongbachdoan/fcj-cd-demo.git
cd amazon-connect

Create a .env file:

cat << EOF > .env
WHISPER_ENDPOINT="whisper-endpoint"
EOF

Execute the following script once:

./scripts/create_deployment_bucket.sh dev

To deploy, execute this script as often as required:
```
./scripts/serverless_deploy.sh dev
```
Set up a contact flow that starts media streaming and passes the following parameters to the ProcessStream Lambda:
- kvsStreamArn: the stream ARN from the contact attribute in Connect.
- kvsStartFragment: the KVS start fragment number from the contact attribute in Connect.
Add any Lambda functions used to the Amazon Connect instance. The ContactId is fetched from the standard request attribute:
```
event.Details.ContactData.ContactId
```
This should start populating an IVR real-time transcript into DynamoDB.
Enable KVS media streaming in your Amazon Connect instance and set a sane retention period for KVS (24 hours minimum during testing).

Step 3: Set Up Amazon Connect Flow

Access your Amazon Connect instance with your account credentials.
Create a flow using the template located at amazon-connect/contact_flow/prsv-aws-connect-flow.json.
Claim a phone number by navigating to Phone numbers > Voice > Toll free > Choose the flow you imported previously.
You're all set! Call your contact center now.

DEMO

https://github.com/user-attachments/assets/dfa474e0-186e-428e-873c-f7f2c991da4d

🎉 Congratulations! 🎉

In this workshop, you have learned how to implement a call center using Amazon Connect and Generative AI. You’ve gained valuable insights into setting up efficient customer support systems that leverage cutting-edge technology to enhance user experience.

Feel free to ask me any questions about this solution! Connect with me on LinkedIn.

Resolve Lambda URL Error - signature not match when using POST/PUT

Vuong Bach Doan — Sat, 31 Aug 2024 05:51:22 +0000

{
   "message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."
}

I. Current issue

💥 A guy with same problem posted a question at this link:
https://repost.aws/questions/QUbHCI9AfyRdaUPCCo_3XKMQ/lambda-function-url-behind-cloudfront-invalidsignatureexception-only-on-post

Describe:

He using CloudFront in front of Lambda Function URL, but he can only using GET method, POST/PUT request is rejected

Analysis problem:

⚠️ Lambda URL have 2 type of authentication:

NONE: anyone with URL can access function
AWS_IAM: require signed-header

If you are using AWS_IAM, all user can’t directly access Lambda URL. It require signed-header as x-amz-content-sha256

II. Solution

Solution 1: Create signed-header with boto3 session

The custom header is add manually from client side.

Note that with this solution, using CloudFront is optional as we do nothing to sign header at CloudFront side, we do it from client side.

CloudFront in this case only for CDN purpose.

Using CloudFront, you can using GET method even though you don’t have x-amz-content-sha256 in header. But if you call directly to Lambda URL, it will reject all METHODs immediately.

Here is how you can sign request:

Step 1. Create boto3 session to sign header

import boto3
from botocore import crt, awsrequest

class SigV4ASign:

    def __init__(self, boto3_session=boto3.Session()):
        self.session = boto3_session

    def get_headers(self, service, region, aws_request_config):
        sigV4A = crt.auth.CrtS3SigV4AsymAuth(self.session.get_credentials(), service, region)
        request = awsrequest.AWSRequest(**aws_request_config)
        sigV4A.add_auth(request)
        prepped = request.prepare()

        return prepped.headers

    def get_headers_basic(self, service, region, method, url):
        sigV4A = crt.auth.CrtS3SigV4AsymAuth(self.session.get_credentials(), service, region)
        request = awsrequest.AWSRequest(method=method, url=url)
        sigV4A.add_auth(request)
        prepped = request.prepare()

        return prepped.headers

Step 2: Using header in our request

from sigv4a_sign import SigV4ASign
import requests

service = 'lambda'
region = '*'
method = 'GET'
url = 'https://4xmze5deqxjjy4ltw2ze3h7gr40tlvcp.lambda-url.us-east-1.on.aws'

headers = SigV4ASign().get_headers_basic(service, region, method, url)
r = requests.get(url, headers=headers)
print(f'status_code: {r.status_code} \nobject text: {r.text}')

Solution 2: Bypass signed-header by create LambdaEdge to assign token at CloudFront

With this solution, all traffic is signed at CloudFront by LambdaEdge no matter who send the request. It means that everyone can call Lambda URL through CloudFront distribution domain.

It only helpful to prevent traffic go directly to Lambda function, but not validate if a user go through CloudFront.

You can implement a solution to sign custom header by follow this document:

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-how-it-works-tutorial.html

Futher Read - Why only GET method can bypass CloudFront to invoke Lambda URL

⁉️ We know that Lambda URL requires all methods to have signed header, but why GET method can go through CloudFront to invoke function?

Here is message when I test with GET method at CloudFront, as you see it includes the Headers, and it included x-amz-content-sha256 .

{
    "message": "Hello from Lambda!",
    "headers": {
        "x-amz-content-sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
        "cloudfront-is-android-viewer": "false",
        "content-length": "0",
        "x-amzn-tls-version": "TLSv1.3",
        "cloudfront-viewer-country": "US",
        "postman-token": "2b178115-dd65-4567-8cfb-07b95ed45d6e",
        "x-amz-source-account": "058264411535",
        "cloudfront-viewer-tls": "TLSv1.3:TLS_AES_128_GCM_SHA256:sessionResumed",
        "x-forwarded-port": "443",
        "x-amz-security-token": "IQoJb3JpZ2luX2VjED0aCXVzLWVhc3QtMSJHMEUCIQDVVeJvqePhgQHPp2rQwVkuB1eNg7lIwpqLd7UpIw4ihwIgBjkJsp9pna8XikOc8Oo0sIW5wsKteL1gc/2PvgVwcu8qtQIIVhAAGgw4NTYzNjkwNTMxODEiDOcCEWCBKujgcCE8KCqSAk3VAG17dDtr7yyyni1Eclx3bOChaK3Lq8ZKhz3LDFZ9KwHErLm5CVFMLg7fHmqHbxYPFxRft6rFJfnePq3XV2x12r8WVNiPMr4rrrNevzB7Qyn2n2CdwOalsFYm7I3fLB4oT5hR7CNN61tzJpupy5XnzbkQJ1HmjnlXRmbQTnkWjpooWuZXZF5hmNKpOAH7+nrn+L97+9xSkYKGs5Yj66BD/TxeoUKBD1extNk8Dz49yvyX3Le9O7rUwWuMof8Qih6aABL8IRQQs5qo2tpuCIODe+nRi2F1j7CJ85YW5MPp2FVvCLOCi0y7ApAiIpzn+g3fQCd830DqEwA1FJ3Mc+eDDy+SZ3j0oDK97lOgAJFtOMQw5JXFtgY6jwGzvIJEeEV29B3yHCpalJciUNzGxWtt2Vw7f1Btv02KgbnGyVzFvs4Dh3Ia6ldLAWevM6ZB/h62870hj0XKBUw5Q/sZR+LqJGDFyD/UfMWb6zSbv6mCc3xtbO/HoxUW7qwNiBGwqDvxvls9383cnpTT8rHPviOGv78QAnNF7qGmKGVPlGWignUvEzYx2Tur8g==",
        "via": "1.1 eb8674b99d3dfcc6867fb20af353442a.cloudfront.net (CloudFront)",
        "x-amzn-tls-cipher-suite": "TLS_AES_128_GCM_SHA256",
        "cloudfront-viewer-asn": "14618",
        "cloudfront-is-desktop-viewer": "true",
        "host": "4xmze5deqxjjy4ltw2ze3h7gr40tlvcp.lambda-url.us-east-1.on.aws",
        "cache-control": "no-cache",
        "cloudfront-viewer-city": "Ashburn",
        "cloudfront-viewer-http-version": "1.1",
        "cloudfront-viewer-address": "54.86.50.139:4863",
        "x-amz-date": "20240830T061644Z",
        "x-forwarded-proto": "https",
        "cloudfront-is-ios-viewer": "false",
        "x-forwarded-for": "54.86.50.139",
        "accept": "*/*",
        "x-amz-source-arn": "arn:aws:cloudfront::058264411535:distribution/E1ZAD106XSWJF5",
        "cloudfront-is-smarttv-viewer": "false",
        "x-amzn-trace-id": "Self=1-66d163cc-0bda717d14f2a84850a6eed2;Root=1-66d163cc-711a9a2d2cad036f1be882a4",
        "cloudfront-is-tablet-viewer": "false",
        "cloudfront-forwarded-proto": "https",
        "accept-encoding": "gzip, deflate, br",
        "x-amz-cf-id": "89c8mF_r2r7fOOOaYREIJ5ZZW4mfurh81xCmzYA5VHbyDTy56i0Crg==",
        "user-agent": "PostmanRuntime/7.41.2",
        "cloudfront-is-mobile-viewer": "false"
    }
}

I did not sign the request before, so we can imagine that CloudFront automatically sign request header for requests that using GET method.

But when I call POST method it show error:

{
    "message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."
}

It clearly that the POST method is not signed by default by CloudFront, we need to create signed token to header.

In this document it saids about why GET is enable by default ( https://community.aws/content/2fuBTcoVg7nnRIVLnqjIsIC8LAi/enhancing-security-for-lambda-function-urls?lang=en ).

To summary, it saids that the solution is for easier access if we enable for GET requests at CloudFront, POST requests still require signed payloads.

In my view, bypass GET can help us with less effort to sign request header, and we know that GET is very common method so if every request need to be signed will take a lot of cost and time. Disadvantage is less secure.

Refs

To read more about solution, I have some helpful link below:

https://github.com/vuongbachdoan/sigv4a-signing-examples/tree/main/python

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-lambda.html

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-how-it-works-tutorial.html

https://community.aws/content/2fuBTcoVg7nnRIVLnqjIsIC8LAi/enhancing-security-for-lambda-function-urls?lang=en

Compare DynamoDB DAX with ElasticCache

Vuong Bach Doan — Tue, 20 Feb 2024 08:43:36 +0000

AWS's DynamoDB, a NoSQL database service, is renowned for its scalability and performance across a range of applications. However, when dealing with read-intensive workloads, frequent interactions with DynamoDB can lead to increased latency and cost. This issue can be mitigated by implementing a caching layer, which stores frequently accessed data closer to the application, thereby reducing database roundtrips. Two popular choices for this purpose are:

Amazon DynamoDB Accelerator (DAX): A fully managed, in-memory caching service tailored specifically for DynamoDB.
Amazon ElasticCache: A versatile caching service that supports various engines such as Memcached and Redis, offering a wider range of applications beyond DynamoDB.

While both options boost performance, they differ in several key aspects:

1. Design and Optimization:

DAX: Specifically designed for DynamoDB, DAX employs the Memcached protocol and optimizes caching strategies for DynamoDB's unique data structures and access patterns, resulting in enhanced efficiency and compatibility with DynamoDB operations.
ElasticCache: Provides a variety of engines, each with its own strengths and trade-offs. Selecting the appropriate engine and configuration requires a careful analysis of data access patterns and performance requirements. It may not integrate as seamlessly with DynamoDB as DAX does.

2. Caching Mechanism:

DAX: Operates in a write-through mode, ensuring data consistency by automatically propagating all writes to the underlying DynamoDB table. It also maintains separate item and query caches, allowing for granular control over caching behavior.
ElasticCache: Supports a range of caching strategies depending on the chosen engine. Some engines, like Memcached, adopt a write-through approach similar to DAX. Others, like Redis, support various strategies such as write-back or read-through, necessitating careful configuration based on specific use cases.

3. Cost and Management:

DAX: Pricing is based on the provisioned cluster size and node type. As a fully managed service, it eliminates the need for manual configuration and maintenance.
ElasticCache: Offers a range of pricing models depending on the chosen engine, instance type, and reserved instances options. It requires more manual configuration and management compared to DAX.

4. Use Cases:

DAX: Ideal for read-intensive workloads specifically targeting DynamoDB. Its close integration and optimized caching strategies make it the go-to choice for maximizing performance and minimizing latency for DynamoDB operations.
ElasticCache: Supports a broader range of use cases beyond DynamoDB, allowing caching for various data sources and applications. It offers more flexibility in caching strategies but may require additional configuration and management effort.

✅Choosing the Right Option:

Key Aspects	Amazon DynamoDB Accelerator (DAX)	Amazon ElasticCache
Design and Optimization	Specifically designed for DynamoDB, employing the Memcached protocol and optimizing caching strategies for DynamoDB's unique data structures and access patterns.	Provides a variety of engines, each with its own strengths and trade-offs. Selecting the appropriate engine and configuration requires a careful analysis of data access patterns and performance requirements.
Caching Mechanism	Operates in a write-through mode, ensuring data consistency by automatically propagating all writes to the underlying DynamoDB table. It also maintains separate item and query caches, allowing for granular control over caching behavior.	Supports a range of caching strategies depending on the chosen engine. Some engines, like Memcached, adopt a write-through approach similar to DAX. Others, like Redis, support various strategies such as write-back or read-through.
Cost and Management	Pricing is based on the provisioned cluster size and node type. As a fully managed service, it eliminates the need for manual configuration and maintenance.	Offers a range of pricing models depending on the chosen engine, instance type, and reserved instances options. It requires more manual configuration and management compared to DAX.
Use Cases	Ideal for read-intensive workloads specifically targeting DynamoDB. Its close integration and optimized caching strategies make it the go-to choice for maximizing performance and minimizing latency for DynamoDB operations.	Supports a broader range of use cases beyond DynamoDB, allowing caching for various data sources and applications. It offers more flexibility in caching strategies but may require additional configuration and management effort.

The choice between DAX and ElasticCache depends on your specific requirements:

If your application is solely focused on enhancing DynamoDB read performance, DAX provides a simpler, optimized, and fully managed solution.
If you require a caching layer for a variety of data sources beyond DynamoDB and need more control over caching strategies, ElasticCache offers greater flexibility.