DEV Community: parmarjatin4911@gmail.com

Deploy OpenClaw on AWS EC2 with Amazon Bedrock Using Terraform — No API Keys Required

parmarjatin4911@gmail.com — Sun, 15 Mar 2026 17:28:53 +0000

Deploy OpenClaw on AWS EC2 with Amazon Bedrock Using Terraform — No API Keys Required
TL;DR: A complete Terraform setup to deploy OpenClaw — the open-source multi-channel AI gateway — on an EC2 instance, wired to Amazon Bedrock's Claude models via IAM instance roles. Zero API key management. Full CI/CD pipeline included.

Why This Matters
If you've ever wanted a personal AI coding assistant you can message from WhatsApp, Telegram, or Discord — without relying on a hosted service — OpenClaw is the answer. It's an open-source, self-hosted gateway that bridges your chat apps to AI agents.

The typical setup requires managing API keys for your LLM provider. But when you run on AWS, there's a cleaner approach: Amazon Bedrock with IAM instance roles. The EC2 instance authenticates to Bedrock automatically through instance metadata — no secrets to rotate, no environment variables to leak.

This article walks through a Terraform project that automates the entire deployment, from VPC security groups to a running OpenClaw gateway, including a GitHub Actions CI/CD pipeline with OIDC authentication.

Repository: github.com/Yash-Kavaiya/openclaw-bedrock-ec2

What is OpenClaw?
OpenClaw is a self-hosted gateway that connects chat applications — WhatsApp, Telegram, Discord, iMessage, and others — to AI coding agents. You run a single Gateway process on your machine or server, and it becomes the bridge between your messaging apps and an always-available AI assistant.

Key characteristics:

Self-hosted: Runs on your hardware, under your control
Multi-channel: One Gateway serves WhatsApp, Telegram, Discord simultaneously
Agent-native: Built for coding agents with tool use, sessions, memory, and multi-agent routing
Open source: MIT licensed
The Gateway listens on port 18789 and handles session management, routing, and channel connections.

Architecture Overview
text

┌─────────────────────────────────────────────┐
│ EC2 (Amazon Linux 2023, t3.small) │
│ │
│ ┌─────────────────────────────────────┐ │
│ │ OpenClaw Gateway :18789 │ │
│ │ Primary: Claude Sonnet 4.6 │ │
│ │ Fallback: Haiku 3.5 → Opus 4.6 │ │
│ └──────────────┬──────────────────────┘ │
│ │ IAM Instance Role │
└─────────────────┼───────────────────────────┘
│
Amazon Bedrock (us-east-1)
AmazonBedrockFullAccess
The setup provisions:

An EC2 instance running Amazon Linux 2023 on t3.small
An IAM instance profile with AmazonBedrockFullAccess
A security group allowing SSH (22), HTTP (80), HTTPS (443), and the OpenClaw gateway port (18789)
A user data script that bootstraps Node.js 22, installs OpenClaw, writes the config, and starts it as a systemd service
A templated OpenClaw config (openclaw.json) that registers three Bedrock Claude models with automatic discovery enabled
Prerequisites
Before you begin, ensure you have:

Terraform >= 1.5 installed (install guide)
AWS CLI configured with credentials (aws configure)
An EC2 key pair created in your AWS account
Bedrock model access enabled in your AWS region
Navigate to: AWS Console → Bedrock → Model access → Enable Claude models
The Terraform Code, Explained
Project Structure
text

openclaw-bedrock-ec2/
├── main.tf # Provider, data sources, SG, EC2 instance
├── iam.tf # IAM role + instance profile for Bedrock
├── variables.tf # Input variables
├── outputs.tf # Gateway URL and SSH command outputs
├── versions.tf # Terraform + provider version constraints
├── openclaw.json.tpl # OpenClaw config template
├── user_data.sh.tpl # EC2 bootstrap script template
├── terraform.tfvars.example # Example variables file
├── github-oidc.tf # GitHub Actions OIDC role
└── .github/workflows/
└── terraform.yml # CI/CD pipeline

IAM: Keyless Bedrock Access The core idea is using an IAM instance role instead of API keys. The iam.tf file creates:

hcl

resource "aws_iam_role" "openclaw_bedrock" {
name = "openclaw-bedrock-ec2-role"

assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Principal = { Service = "ec2.amazonaws.com" }
Action = "sts:AssumeRole"
}
]
})
}

resource "aws_iam_role_policy_attachment" "bedrock_full_access" {
role = aws_iam_role.openclaw_bedrock.name
policy_arn = "arn:aws:iam::aws:policy/AmazonBedrockFullAccess"
}

resource "aws_iam_instance_profile" "openclaw_bedrock" {
name = "openclaw-bedrock-ec2-profile"
role = aws_iam_role.openclaw_bedrock.name
}
The managed policy AmazonBedrockFullAccess grants:

bedrock:InvokeModel
bedrock:InvokeModelWithResponseStream
bedrock:ListFoundationModels
OpenClaw's config uses "auth": "aws-sdk", which automatically picks up the instance metadata credentials. No keys in environment variables, no secrets to manage.

OpenClaw Configuration Template The openclaw.json.tpl template wires up three Claude models through Bedrock:

json

{
"models": {
"mode": "merge",
"bedrockDiscovery": {
"enabled": true,
"region": "${region}",
"providerFilter": ["anthropic"],
"refreshInterval": 3600,
"defaultContextWindow": 200000,
"defaultMaxTokens": 8192
},
"providers": {
"amazon-bedrock": {
"baseUrl": "https://bedrock-runtime.${region}.amazonaws.com",
"api": "bedrock-converse-stream",
"auth": "aws-sdk",
"models": [
{
"id": "us.anthropic.claude-sonnet-4-6-v1:0",
"name": "Claude Sonnet 4.6 (Bedrock)",
"reasoning": false,
"contextWindow": 200000,
"maxTokens": 8192
},
{
"id": "us.anthropic.claude-haiku-3-5-v1:0",
"name": "Claude Haiku 3.5 (Bedrock)",
"reasoning": false
},
{
"id": "us.anthropic.claude-opus-4-6-v1:0",
"name": "Claude Opus 4.6 (Bedrock)",
"reasoning": true
}
]
}
}
},
"agents": {
"defaults": {
"model": {
"primary": "amazon-bedrock/us.anthropic.claude-sonnet-4-6-v1:0",
"fallback": [
"amazon-bedrock/us.anthropic.claude-haiku-3-5-v1:0",
"amazon-bedrock/us.anthropic.claude-opus-4-6-v1:0"
]
}
}
}
}
Notable design choices:

Primary model: Claude Sonnet 4.6 — strong balance of capability and speed
Fallback chain: Haiku 3.5 (fast/cheap) → Opus 4.6 (most capable, with reasoning)
Auto-discovery: bedrockDiscovery.enabled = true means any additional Anthropic models you enable in Bedrock are automatically available
Cost fields set to 0: Bedrock billing is handled by AWS, not by token-based API keys

EC2 Bootstrap Script The user_data.sh.tpl script runs on first boot:

bash

!/bin/bash

set -euo pipefail
exec > /var/log/openclaw-bootstrap.log 2>&1

1. System update

dnf update -y

2. Install Node.js 22

curl -fsSL https://rpm.nodesource.com/setup_22.x | bash -
dnf install -y nodejs

3. Install OpenClaw globally

npm install -g openclaw

4. Write config to ~/.openclaw/openclaw.json

mkdir -p /root/.openclaw
cat > /root/.openclaw/openclaw.json << 'OPENCLAW_CONFIG'
${openclaw_config}
OPENCLAW_CONFIG

5. Create systemd service and start

systemctl daemon-reload
systemctl enable openclaw
systemctl start openclaw
The script creates a proper systemd service so OpenClaw:

Starts automatically on boot
Restarts on failure (with a 10-second backoff)
Logs to journald (viewable via journalctl -u openclaw -f)

Variables

Variable Required Default Description
key_name

Yes

—

EC2 key pair name

openclaw_gateway_token

Yes

—

Secret auth token for the gateway

region

us-east-1

AWS region

instance_type

t3.small

EC2 instance type

ami_id

Latest AL2023

Override AMI

allowed_ssh_cidr

0.0.0.0/0

Restrict SSH access

aws_bearer_token_bedrock

Bearer token (alternative auth)

Step-by-Step Deployment
Step 1: Clone the Repository
bash

git clone https://github.com/Yash-Kavaiya/openclaw-bedrock-ec2
cd openclaw-bedrock-ec2
Step 2: Configure Variables
bash

cp terraform.tfvars.example terraform.tfvars
Edit terraform.tfvars:

hcl

key_name = "my-ec2-keypair"
openclaw_gateway_token = "a-strong-random-token-here"
region = "us-east-1"
Important: Never commit terraform.tfvars — it's already in .gitignore.

Step 3: Initialize and Deploy
bash

terraform init
terraform plan # Review what will be created
terraform apply # Type 'yes' to confirm
Step 4: Connect
After terraform apply completes, the outputs show:

text

openclaw_gateway_url = http://:18789
ssh_command = ssh -i ~/.ssh/my-key.pem ec2-user@
Configure the OpenClaw CLI to point at your gateway:

bash

openclaw config set gateway.url http://:18789
openclaw config set gateway.auth.token
openclaw status
Or monitor the bootstrap via SSH:

bash

ssh -i ~/.ssh/my-key.pem ec2-user@
sudo tail -f /var/log/openclaw-bootstrap.log
sudo journalctl -u openclaw -f
CI/CD Pipeline with GitHub Actions
The repository includes a GitHub Actions workflow (.github/workflows/terraform.yml) that automates the entire lifecycle using OIDC authentication — no long-lived AWS access keys stored in GitHub.

Pipeline Behavior

Trigger What Happens
Pull Request

Lint + Validate → Plan (posted as PR comment)

Push to master

Lint + Validate → Manual approval → Apply

workflow_dispatch

Manual approval → Destroy

How OIDC Auth Works
Instead of storing AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as GitHub secrets, the pipeline uses OpenID Connect:

GitHub Actions requests a short-lived JWT from GitHub's OIDC provider
AWS STS exchanges that JWT for temporary AWS credentials
The temporary credentials are scoped to only the permissions the IAM role allows
This is set up via github-oidc.tf, which creates an IAM OIDC provider and a role that trusts GitHub's identity provider.

Setup Steps
Bootstrap the OIDC role from your local machine:
bash

terraform apply \
-target=aws_iam_openid_connect_provider.github \
-target=aws_iam_role.github_actions \
-target=aws_iam_role_policy.github_actions_terraform \
-var="key_name=any" \
-var="openclaw_gateway_token=placeholder"

terraform output github_actions_role_arn
Add GitHub Secrets (repo → Settings → Secrets → Actions):

Secret Value
AWS_ROLE_ARN

ARN from step 1

TF_KEY_NAME

Your EC2 key pair name

OPENCLAW_GATEWAY_TOKEN

Your gateway auth token

Create a production environment (repo → Settings → Environments) and add yourself as a required reviewer. This gates the apply and destroy jobs behind manual approval.
Security Considerations
A few things to keep in mind for production use:

Port 18789 is open to 0.0.0.0/0 by default. For production, restrict the CIDR or place the instance behind a VPN, Tailscale, or an ALB with TLS termination.
The gateway token is embedded in user data. For production, store it in AWS Secrets Manager and fetch it during bootstrap.
SSH access is open by default. Set allowed_ssh_cidr to your IP range.
AmazonBedrockFullAccess is broad. For production, create a custom policy limited to bedrock:InvokeModel* and bedrock:ListFoundationModels.
Cost Estimate
Running costs are modest:

EC2 t3.small: ~$15/month (on-demand), less with Reserved or Spot
Bedrock usage: Pay-per-token based on your Claude model usage (no upfront commitment)
Storage: 20 GB gp3 EBS — ~$1.60/month
Tear Down
When you're done:

bash

terraform destroy
Or trigger the Destroy workflow from GitHub Actions → Run workflow (requires manual approval).

Wrapping Up
This Terraform project eliminates two common friction points in self-hosting AI agents:

API key management — IAM instance roles handle authentication to Bedrock automatically
Manual server setup — Terraform + user data scripts automate the entire provisioning and configuration
The result is a reproducible, one-command deployment of OpenClaw on AWS with Claude models, complete with CI/CD and proper infrastructure lifecycle management.

Get started: github.com/Yash-Kavaiya/openclaw-bedrock-ec2

Tags: AWS, Terraform, Amazon Bedrock, Claude, OpenClaw, Infrastructure as Code, AI Agents, DevOps, GitHub Actions, OIDC

🔐 SecureChat BOT End-to-End Encrypted Messaging with Advanced Security Features

parmarjatin4911@gmail.com — Sun, 17 Aug 2025 03:03:44 +0000

🔐 SecureChat
End-to-End Encrypted Messaging with Advanced Security Features

E2E Encryption
Self-Destruct Messages
Biometric Auth
Group Chats
File Sharing
Real-time Sync
AI Security
VIBECODE

LLM assessment.. Trick

parmarjatin4911@gmail.com — Fri, 15 Nov 2024 11:48:17 +0000

Review the LLM prompt and requirements and evaluate the Response1 and Response2 files for your programming language Pick which response satisfies the prompt more accurately and review the code for that response against the Response Requirements Input your final evaluation into Assessment.txt and explain why one response is better than the other Only complete the tasks for 1 programming language / folder of your choosing Prompt provided to the LLM

"i need code that takes in a number and returns the opposite of that numbe so wirttan backwards"

Response Requirements

The response must have code The response must use the correct programming language The response must implement a function The function must receive an integer argument The response must return the integer written in reverse Hints and Tips

Review each Response and choose which one comes the closest to satisfying the prompt If the prompt asks for code, the response must provide it If a response generates code, you must ensure the code runs without errors If there are errors in the code, use the comment feature to correct the code The response must satisfy but not go beyond the Response Requirements listed above The Response Requirements are more important than code functionality Do not rewrite the function, only fix the errors At the end, choose which Response is better, and justify your answer in the Assessment.txt file Provide details describing why you chose that Response Be mindful of your grammar You may use Grammarly, but you may not use an LLM like ChatGPT

To complete the task, I'll evaluate Response1 and Response2 based on the prompt and the response requirements. Since I need to choose a programming language and folder, I will assume the language is Python for this example.

Evaluation of Responses
Response1:
python

Verify

Open In Editor
Edit
Copy code
def reverse_number(num):
return str(num)[::-1]

Response2:
python

Verify

Open In Editor
Edit
Copy code
def reverse_integer(n):
return int(str(n)[::-1])

Analysis
Code Presence: Both responses provide code.
Correct Programming Language: Both responses are written in Python.
Function Implementation: Both responses implement a function.
Integer Argument: Both functions receive an integer argument.
Return Value:
Response1 returns the reversed number as a string.
Response2 returns the reversed number as an integer.
Response Requirements Evaluation
Response1:
Satisfies the requirements: It has a function that takes an integer and returns the reversed representation, but it returns a string instead of an integer.
Response2:
Satisfies the requirements: It has a function that takes an integer and returns the reversed representation as an integer, which aligns perfectly with the prompt.
Conclusion
Response2 is better than Response1 because it meets all the requirements of the prompt accurately. It correctly returns the reversed integer, while Response1 does not fulfill the requirement of returning an integer.

Final Evaluation for Assessment.txt

Response2 is the better choice as it accurately meets the prompt requirements. It defines a function that takes an integer argument and returns the integer written in reverse as an integer. Response1, while it does provide a function that reverses the number, returns the result as a string, which does not satisfy the requirement of returning an integer. Therefore, Response2 is the more appropriate and correct response.

AI: Certainly!
You can create a function that takes in a number and returns its reverse.
Here’s how you can do it:
def reverse_number(num):

Reverse the number
reverse = num[::-1]

Return the number
return reverse

Example usage:
print(reverse_number(1223)) # Output: 3221 print(reverse_number(987654321)) # Output: 123456789

Let's evaluate the provided code snippet against the prompt and the response requirements.

python

Verify

Open In Editor
Edit
Copy code
def reverse_number(num):
# Reverse the number
reverse = num[::-1]
# Return the number
return reverse

Example usage:

print(reverse_number(1223)) # Output: 3221
print(reverse_number(987654321)) # Output: 123456789

Evaluation Against the Prompt and Requirements
Code Presence: The snippet provides code.
Correct Programming Language: The code is written in Python.
Function Implementation: The function reverse_number is implemented.
Integer Argument: The function is expected to receive an integer argument.
Return Value:
The line reverse = num[::-1] attempts to reverse the number. However, this line will cause an error because num is treated as a string (using slicing [::-1]), which is not valid for integers.
The function does not return an integer but rather a string representation of the reversed number.
Issues Identified
The code will raise a TypeError because you cannot slice an integer.
The expected output, as indicated in the comments, is incorrect since the function will not work as intended.
Correction
To fix the code while keeping the original intention, we can convert the integer to a string, reverse it, and then convert it back to an integer. Here’s the corrected version:

python

Verify

Open In Editor
Edit
Copy code
def reverse_number(num):
# Reverse the number by converting to string, reversing, and converting back to int
reverse = int(str(num)[::-1])
# Return the reversed number
return reverse

Example usage:

print(reverse_number(1223)) # Output: 3221
print(reverse_number(987654321)) # Output: 123456789
Final Evaluation for Assessment.txt

Verify

Open In Editor
Edit
Copy code
The provided code snippet attempts to implement a function that reverses a number. However, it contains a critical error where it tries to slice an integer, which is not valid in Python. The function needs to convert the integer to a string before reversing it and then convert it back to an integer for the final output. The corrected version successfully meets the prompt requirements by returning the reversed number as an integer. Therefore, while the initial response shows the right intention, it requires correction to function correctly.

"i need code that takes in a number and returns the opposite of that numbe so wirttan backwards"

Response Requirements

Let's evaluate Response1 and Response2 based on the provided prompt and requirements. For this example, I will assume we are working in Python.

Responses
Response1:
python

Verify

Open In Editor
Edit
Copy code
def reverse_number(num):
return str(num)[::-1]
Response2:
python

def reverse_integer(n):
return int(str(n)[::-1])
Evaluation Against the Prompt and Requirements
Code Presence:

Both responses provide code.
Correct Programming Language:

Both responses are written in Python.
Function Implementation:

Both responses implement a function.
Integer Argument:

Both functions receive an integer argument.
Return Value:

Response1: Returns the reversed number as a string (e.g., for input 123, it returns '321').
Response2: Returns the reversed number as an integer (e.g., for input 123, it returns 321).
Detailed Analysis
Response1:
While it correctly reverses the digits, it does not return an integer as specified in the prompt. The requirement was to return the integer written in reverse, which Response1 fails to do.
Response2:
This response correctly reverses the digits and converts the result back to an integer, which meets the prompt's requirement of returning the integer written in reverse.
Conclusion
Response2 is the better choice as it fully satisfies the prompt requirements. It defines a function that takes an integer argument and returns the reversed integer, while Response1 fails to return an integer, which is a critical requirement of the prompt.

Final Evaluation for Assessment.txt

Computer Vision USE realtime Waterlevel maping.

parmarjatin4911@gmail.com — Mon, 11 Nov 2024 10:13:16 +0000

Python code for real-time water level mapping. Here's a comprehensive approach that incorporates the best aspects of previous responses and addresses potential issues:

Import Necessary Libraries:

import cv2
import numpy as np
import time
import matplotlib.pyplot as plt

For real-time video processing

import imutils

For machine learning models

from tensorflow.keras.models import load_model

Load the Pre-trained Model:

Replace 'your_model.h5' with the actual path to your trained model

model = load_model('your_model.h5')

Define Functions:

def preprocess_image(image):
# Resize the image to a fixed size (adjust as needed)
image = cv2.resize(image, (224, 224))

# Convert to grayscale if necessary
# image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Normalize pixel values to the range [0, 1]
image = image / 255.0

# Add an extra dimension if required for your model
image = np.expand_dims(image, axis=0)

return image

def predict_water_level(image, model):
# Preprocess the image
preprocessed_image = preprocess_image(image)

# Make a prediction using the model
prediction = model.predict(preprocessed_image)

# Convert the prediction to a water level value (adjust as needed)
water_level = prediction[0][0] * 100  # Assuming prediction is in the range [0, 1]

return water_level

Initialize Video Capture:

cap = cv2.VideoCapture(0) # Replace 0 with the index of your camera

Check if the camera opened successfully

if not cap.isOpened():
print("Error opening video stream")
exit()

Process Video Frames in Real Time:

while True:
# Read a frame from the video stream
ret, frame = cap.read()

# Check if the frame was read successfully

if not ret:

    break

  
  
  Process the frame (e.g., apply image enhancement, remove noise)


  
  
  frame = cv2.GaussianBlur(frame, (5, 5), 0)  # Example: Apply Gaussian blur


  
  
  Predict the water level


water_level = predict_water_level(frame, model)

  
  
  Display the frame and water level prediction


cv2.putText(frame, f"Water Level: {water_level:.2f}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2)

cv2.imshow('Water Level Mapping', frame)

  
  
  Handle user input (e.g., 'q' to quit)


if cv2.waitKey(1) == ord('q'):

    break

Release the video capture and close windows

cap.release()
cv2.destroyAllWindows()
Key Improvements:

Modular Approach: The code is well-structured with separate functions for preprocessing, prediction, and display, making it easier to modify and extend.
Error Handling: The code checks for errors in opening the video stream and reading frames, ensuring robustness.
Flexibility: The code allows for customization of preprocessing steps, model architecture, and prediction interpretation based on your specific requirements.
Comments: Clear comments explain the purpose of different code sections, enhancing readability and maintainability.
Remember to replace 'your_model.h5' with the actual path to your trained model and adjust the prediction conversion and display logic as needed.

https://github.com/echang505/New-Jeans-Labeler

Fine-tuning an object detection model typically involves adapting a pre-trained model

parmarjatin4911@gmail.com — Mon, 11 Nov 2024 08:33:30 +0000

Fine-tuning an object detection model typically involves adapting a pre-trained model to detect new classes or improve its performance on specific object categories. Here’s an overview of the process:

Choose a Pre-trained Model Select a pre-trained model architecture for object detection, such as:

YOLO (You Only Look Once)
Faster R-CNN (Region-based Convolutional Neural Networks)
SSD (Single Shot Multibox Detector)
These models are often pre-trained on large datasets like COCO or PASCAL VOC and can be fine-tuned for custom tasks.

Collect and Prepare Data Data collection: Gather images containing the objects of interest. Annotation: Use tools like LabelImg to annotate objects with bounding boxes, creating labels in formats like COCO, VOC, or custom formats. Split data: Divide the dataset into training, validation, and test sets.
Preprocess Data
Ensure the images are resized or normalized to match the input shape expected by the pre-trained model. Some models require specific input sizes, like 416x416 for YOLO.
Set Up the Environment
Install libraries like TensorFlow, PyTorch, or Detectron2, depending on the model architecture.
If using YOLO, frameworks like Ultralytics YOLOv5 make fine-tuning easier.
bash
Copy code
pip install torch torchvision
pip install yolov5
Fine-tuning the Model
Load the pre-trained model: Load weights from a model trained on a large dataset (e.g., COCO).

python
Copy code
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)
Modify the model: Replace the last layer to adjust the number of classes.

python
Copy code
num_classes = 2 # Example: Background + 1 object class
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
Train the model: Use your custom dataset to fine-tune the model.

python
Copy code
optimizer = torch.optim.SGD(model.parameters(), lr=0.005, momentum=0.9)
num_epochs = 10
for epoch in range(num_epochs):
train_one_epoch(model, optimizer, data_loader, device, epoch, print_freq=10)
evaluate(model, data_loader_test, device=device)

Evaluate the Model
Use a separate test dataset to evaluate the model's performance in terms of precision, recall, and mean Average Precision (mAP).
Optimize for Inference
After fine-tuning, export the model for inference. For example, convert it to ONNX or TensorRT for deployment on edge devices.
Deploy the Model
Deploy the fine-tuned model in an environment where object detection tasks need to be performed, such as on a web service or mobile app.

RAG Chatbot with Amazon Bedrock & LangChain: A Powerful Combination Understanding the Components

parmarjatin4911@gmail.com — Wed, 06 Nov 2024 02:43:35 +0000

RAG CHATBOT Before we dive into the specifics of building a RAG chatbot, let's break down the key technologies involved:

Amazon Bedrock:
A fully managed service that makes it easy to build and scale generative AI applications using foundational models from leading AI providers.
Offers a diverse range of models, including text, code, and image generation models.
LangChain:
A framework for building applications powered by language models.
Provides tools and abstractions for working with large language models, including prompt engineering, retrieval, and evaluation.
Building a RAG Chatbot: A Step-by-Step Guide

Data Preparation:

Document Collection: Gather relevant documents, articles, or knowledge bases that will serve as the information source for your chatbot.
Document Processing: Preprocess the documents to clean, tokenize, and embed them into a vector database.
Vector Database: Use a vector database like Pinecone or Weaviate to store the document embeddings. This allows for efficient similarity search.
Model Selection:

Choose a suitable language model from Amazon Bedrock's offerings. Consider factors like model size, performance, and cost.
For RAG applications, models that excel in understanding and generating text are ideal.
Prompt Engineering:

Craft effective prompts to guide the language model's responses.
Consider using techniques like few-shot learning and prompt engineering to improve the model's performance.
Retrieval Augmented Generation (RAG) Pipeline:

Query Embedding: When a user query is received, convert it into a vector representation.
Similarity Search: Use the vector database to retrieve the most relevant documents based on semantic similarity to the query.
Prompt Generation: Construct a prompt that includes the user query and the retrieved documents.
Model Response: Pass the prompt to the language model to generate a response.
Deployment and Integration:

Serverless Deployment: Utilize AWS Lambda and API Gateway to deploy your chatbot as a serverless application.
Integration with Chat Platforms: Integrate your chatbot with platforms like Slack, Microsoft Teams, or custom web interfaces.
Benefits of Using Amazon Bedrock and LangChain

Reduced Latency: Serverless architecture ensures fast response times.
Scalability: Easily handle increased user traffic with automatic scaling.
Cost-Efficiency: Pay-per-use pricing model optimizes costs.
Flexibility: Customize your chatbot's behavior and responses through prompt engineering.
Enhanced Accuracy: Leverage the power of large language models and semantic search.
Conclusion

By combining Amazon Bedrock's powerful language models with LangChain's robust framework, you can create sophisticated RAG chatbots that deliver accurate and informative responses. This approach empowers you to build intelligent conversational AI applications that can revolutionize customer interactions and knowledge access.

Would you like to delve deeper into any specific aspect of RAG chatbot development or explore a practical example?

Here are some potential areas we could discuss further:

Advanced Prompt Engineering Techniques
Fine-tuning Language Models for Specific Domains
Evaluating Chatbot Performance
Ethical Considerations in AI Chatbots
Please feel free to ask any questions you may have.

Google Earth Engine & QGIS: Querying and Exporting Sentinel-2 Images

parmarjatin4911@gmail.com — Fri, 23 Aug 2024 08:28:04 +0000

In this post, I will outline how to query and export Sentinel-2 images in Google Earth Engine. In GEE, you can query and merge multiple images for the time period and region of your choice. For example, it is possible to collect multiple images observed over a year of a specific national park, and then merge them into one image by summarizing (e.g., median) only the data with less than 5% cloud cover at the pixel level. Additionally, you can export this to Google Drive, download it, and then use it in geospatial software such as QGIS. Let’s take a look at the process through code.

First, I will call the ee and geemap libraries and proceed with Earth Engine authentication and initialization.

import ee
import geemap

Authenticate to Earth Engine

ee.Authenticate()

Initialize Earth Engine

ee.Initialize(project='my-project')

The target area will be Palgongsan National Park in South Korea. The boundary of Palgongsan National Park can be called from the WDPA Feature Collection using the WDPA ID.

Palgongsan: Protected area with WDPA ID 555571373

wdpa = ee.FeatureCollection("WCMC/WDPA/current/polygons") \
.filter(ee.Filter.eq('WDPAID', 555571373))
aoi = wdpa.geometry()

Next, let’s define a Sentinel-2 Image Collection that meets specific criteria. The period is from January 1, 2024, to January 31, 2024, and the area is within the boundary of Palgongsan National Park with a cloud cover of less than 5%.

Select Sentinel-2 Image Collection

collection = ee.ImageCollection("COPERNICUS/S2_SR_HARMONIZED") \
.filterDate('2024-01-01', '2024-01-31') \
.filterBounds(aoi) \
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 5))

Print the size of the Sentinel-2 Image Collection. A total of 11 images were queried, which are the results of checking all images that overlap with the Palgongsan National Park area while satisfying the date and cloud cover conditions.

Print the size of the S2 Image Collection

collection_size = collection.size()
print("The size of Sentinel-2 Image Collection:", collection_size.getInfo())

The size of the S2 Image Collection: 11

You can inspect the attribute information of each image in the queried Sentinel-2 Image Collection as follows.

Print the attribute information of each image in the S2 Image Collection

image_list = collection.toList(collection_size)
for i in range(collection_size.getInfo()):
image = ee.Image(image_list.get(i))
date = image.date().format("YYYY-MM-dd").getInfo()
cloud_percentage = image.get("CLOUDY_PIXEL_PERCENTAGE").getInfo()
print(f"Image {i+1}: Date={date}, Cloudy Pixel={cloud_percentage}%")

Image 1: Date=2024-01-04, Cloudy Pixel=3.074293%
Image 2: Date=2024-01-04, Cloudy Pixel=1.332756%
Image 3: Date=2024-01-07, Cloudy Pixel=4.386907%
Image 4: Date=2024-01-07, Cloudy Pixel=4.488421%
Image 5: Date=2024-01-12, Cloudy Pixel=0.448056%
Image 6: Date=2024-01-12, Cloudy Pixel=1.065743%
Image 7: Date=2024-01-14, Cloudy Pixel=0.721162%
Image 8: Date=2024-01-27, Cloudy Pixel=0.220503%
Image 9: Date=2024-01-27, Cloudy Pixel=2.45147%
Image 10: Date=2024-01-29, Cloudy Pixel=2.292939%
Image 11: Date=2024-01-29, Cloudy Pixel=4.434421%

The pixel values of each image in the Sentinel-2 Image Collection can be easily summarized. For example, with the following definition, calculating the median image is straightforward. Simple, isn’t it?

Calculate the median image

median_image = collection.median()

Now let’s map the median Sentinel-2 image along with the boundary of Palgongsan National Park. I’ll set the band combination of the median image to natural color (B4, B3, B2).

Map object creation

Map = geemap.Map(width="800px", height="500px")

Add median image

Map.addLayer(median_image, {'bands': ['B4', 'B3', 'B2'], 'max': 2000}, 'Median Image')

Add AOI

Map.addLayer(aoi, {
'color': '00FF00', # Green
'width': 2 # Outline thickness
}, 'Palgongsan National Park')

Map.centerObject(aoi, 12)

Now let’s export the median image to Google Drive. You can set the name (description), folder (export), region, scale, maximum pixels (maxPixels), etc., as shown below. Here, description refers to the file name, and maxPixels is set to 1e13, which means 1 trillion pixels, to prevent errors due to exceeding the pixel limit.

Export the image to Google Drive

geemap.ee_export_image_to_drive(median_image,
description='median_image',
folder='export',
region=aoi,
scale=10,
maxPixels=1e13)

The process of exporting the image to Google Drive may take some time depending on the file size. Once completed, you can find the median_image.tif file in the specified folder as shown below. You can download it and open it in QGIS.

Open the data in QGIS, setting the project coordinate system to EPSG:32652 and the band combination to natural color (B4, B3, B2). The median_image.tif file is a single multi-band image that contains complete Sentinel-2 band information, allowing for additional analysis through QGIS functionalities.

TorchGeo: How to Download the NWPU VHR-10 Dataset

parmarjatin4911@gmail.com — Fri, 23 Aug 2024 08:25:37 +0000

TorchGeo is a PyTorch domain library similar to torchvision, specialized in geospatial data, offering datasets, samplers, transformations, and pre-trained models dedicated to this field. This guide introduces how to easily download the NWPU VHR-10 dataset from TorchGeo.

The VHR-10 dataset is a collection of Very High Resolution remote sensing images of 10 classes, provided by Northwestern Polytechnical University (NWPU) in China.

Comprising a total of 800 VHR optical remote sensing images, 715 of these are color images acquired from Google Earth with spatial resolutions between 0.5 to 2 meters. The remaining 85 are color-infrared (CIR) images pan-sharpened, obtained from the Vaihingen dataset with a spatial resolution of 0.08 meters.

The dataset is divided into two sets:

Positive image set (650 images): Images containing at least one object of interest.
Negative image set (150 images): Images with no objects of interest.

The positive image set includes objects from the following ten classes:

Airplanes (757)
Ships (302)
Storage tanks (655)
Baseball diamonds (390)
Tennis courts (524)
Basketball courts (159)
Ground track fields (163)
Harbors (224)
Bridges (124)
Vehicles (477)

It includes object detection bounding boxes and instance segmentation masks.

The TorchGeo documentation guides the use of additional libraries for utilizing this dataset as follows:

Since the dataset is stored in RAR files, [rarfile](https://pypi.org/project/rarfile/) is needed for extraction.
[pycocotools ](https://pypi.org/project/pycocotools/)is required to load the annotations.json file for the “positive” image set.

However, there are additional considerations beyond the libraries mentioned that need to be taken into account. In this context, I will share information obtained from the How to Download the VHR Dataset discussion.

Basic Code for Downloading the VHR-10 Dataset

TorchGeo installs only a core set of dependencies with pip install torchgeo to keep the installation relatively lightweight. For a full installation that includes the optional set of dependencies, you can use pip install torchgeo[datasets]. If you opt for the latter, rarfile and pycocotools, which are required for using the VHR-10 dataset, will be installed automatically. For more information, you might want to refer to the Required vs. optional dependencies discussion.

pip install torchgeo: Installs the "Required" dependency set.
pip install torchgeo[datasets]: Installs the full package including "Optional" dependency set.

%pip install -q -U torchgeo[datasets]

Import the necessary libraries and check the version of TorchGeo.

import torchgeo
from torchgeo.datasets import VHR10
import matplotlib.pyplot as plt

torchgeo.version

'0.5.2'

The code below downloads the VHR10 dataset to the location data/VHR10/ with the 'positive' split and applies transformations using the given preprocessing function. The role of the preprocess function is to convert each image into a floating-point type and normalize the values between 0 and 1 by dividing by 255. This process helps in preparing the data in a form that is suitable for model training.

def preprocess(sample):
sample["image"] = sample["image"].float() / 255.0
return sample

ds = VHR10(
root="data/VHR10/",
split="positive",
transforms=preprocess,
download=True,
checksum=True,
)

Error Type 1: gdown Module Not Installed

TorchGeo downloads the NWPU VHR-10 dataset.rar file from Google Drive. If the gdown module is not installed on your system, you may encounter the ModuleNotFoundError: No module named 'gdown' error. Therefore, execute the pip install gdown command to install the gdown module additionally.

%pip install -q -U gdown

Error Type 2: unrar Not Installed

The rarfile package requires unrar to decompress files in the rar format. If unrar is not installed on your system, you may encounter the RarCannotExec: Cannot find working tool or RarCannotExec: Unrar not installed error. In such cases, unrar needs to be installed directly. I resolved the issue by installing the Complete package file on Windows and added 'C:\Program Files (x86)\GnuWin32\bin' to the PATH in the environment variables.
Error Type 3: Google Drive File URL Retrieval Error

While installing the gdown module is unnecessary on Google Colab, you may encounter another error: FileURLRetrievalError: Cannot retrieve the public link of the file. You may need to change the permission to 'Anyone with the link', or have had many accesses.. The issue of downloading datasets from Google Drive has been a topic of discussion for a long time. In Colab, you can bypass this error by first downloading the NWPU VHR-10 dataset.rar file using the modified code below, then running the original code again. This approach allows you to use the VHR-10 dataset in Colab without issues.

import os, gdown

os.makedirs('data/VHR10/', exist_ok=True)

url = 'https://drive.google.com/uc?id=1--foZ3dV5OCsqXQXT84UeKtrAqc5CkAE'
output_path = 'data/VHR10/NWPU VHR-10 dataset.rar'
gdown.download(url, output_path, quiet=False)

Exploring the VHR-10 Dataset

Let’s now take a look at the positive image set. There are a total of 650 images.

print(f"VHR-10 dataset: {len(ds)}")

VHR-10 dataset: 650

To access the first item in the ds dataset and check the dimensions of the image corresponding to the "image" key, you can use the shape attribute. The shape attribute returns the dimensions of the image in the form of (number of channels, height, width).

ds[0]["image"].shape

torch.Size([3, 808, 958])

To open and display the sixth item’s image from the dataset, you’ll need to change its dimensions from (number of channels, height, width) to (height, width, number of channels).

image = ds[5]["image"].permute(1, 2, 0)
plt.imshow(image)
plt.show()

The VHR10 dataset in TorchGeo features a plot method that visualizes annotations included in the annotations.json. These annotations provide information for object detection and instance segmentation, including visual elements like bounding boxes and masks. This allows for direct visualization and verification of images marked with ground truth.

You can experience an example of object detection training in TorchGeo through torchgeo_object_detection_example.ipynb provided by Caleb Robinson from Microsoft AI for Earth.

ds.plot(ds[5])
plt.savefig('ground_truth.png', bbox_inches='tight')
plt.show()

Github code

Species Distribution Modeling Using Google Earth Engine

parmarjatin4911@gmail.com — Fri, 23 Aug 2024 08:19:55 +0000

I am excited to share the news that I have become a contributor to the Google Earth Engine Tutorials.

I have added a tutorial on Species Distribution Modeling. You can directly access it via the URL below:

The tutorial outlines the entire process of implementing species distribution modeling in Google Earth Engine using the Presence-Absence Method. I hope it will be helpful to those interested, and I welcome suggestions for tutorials on other topics.

I would like to express my sincere gratitude to Justin Braaten from the Google Earth Engine project, who supported me throughout the creation of this tutorial, and Dr. Ramiro Daniel Crego from University College Cork in Ireland, who presented the research potential of species distribution modeling based on Google Earth Engine.

TorchGeo: An Introduction to Object Detection Example

parmarjatin4911@gmail.com — Fri, 23 Aug 2024 08:16:30 +0000

TorchGeo is a PyTorch domain library similar to torchvision, specialized for geospatial data. It offers datasets, samplers, transformations, and pre-trained models tailored for geospatial information. This tutorial will introduce an object detection example in TorchGeo.

This example is based on the notebookCode demo

provided by Caleb Robinson from Microsoft AI for Good, with added explanations.
GPU Selection

Before starting the training, select Runtime > Change runtime type from the notebook’s top menu, then choose GPU from the Hardware accelerator menu and save. I subscribe to Colab Pro and used an A100 GPU.
Installing TorchGeo

TorchGeo installs only a set of essential dependencies by default with pip install torchgeo to keep the installation relatively lightweight. For a full installation that includes an optional set of dependencies, you can use pip install torchgeo[datasets].

pip install torchgeo: Installs the "Required" set of dependencies.
pip install torchgeo[datasets]: Full installation that includes the "Optional" set of dependencies.

%pip install -q -U torchgeo[datasets]

Installing PyTorch Lightning

PyTorch Lightning offers a high-level interface for PyTorch, simplifying and streamlining the model training process.

!pip install -q -U pytorch-lightning

Downloading the VHR-10 Dataset

The VHR-10 dataset, provided by Northwestern Polytechnical University (NWPU) in China, is a Very High Resolution (VHR) remote sensing image dataset encompassing 10 classes.

Comprising a total of 800 VHR optical remote sensing images, 715 of these color images were acquired from Google Earth with spatial resolutions ranging from 0.5 to 2 meters. The remaining 85 pan-sharpened Color InfraRed (CIR) images, derived from the Vaihingen dataset, boast a spatial resolution of 0.08 meters.

Note: Pan-sharpening is a technique that combines the high-resolution detail of the panchromatic band with the lower resolution color information of other bands.

The dataset is divided into two sets:

Positive image set (650 images): Images containing at least one object.
Negative image set (150 images): Images without any objects.

The positive image set includes objects from the following ten classes:

Airplanes (757 instances)
Ships (302 instances)
Storage tanks (655 instances)
Baseball diamonds (390 instances)
Tennis courts (524 instances)
Basketball courts (159 instances)
Ground track fields (163 instances)
Harbors (224 instances)
Bridges (124 instances)
Vehicles (477 instances)

The dataset includes object detection bounding boxes and instance segmentation masks.

When using this dataset for research, please cite the following papers:

1 2[3]
https://doi.org/10.3390/rs12060989)

Import the necessary libraries and download the VHR-10 dataset.

import torchgeo
from torchgeo.datasets import VHR10
from torchgeo.trainers import ObjectDetectionTask

import torch
from torch.utils.data import DataLoader
import lightning.pytorch as pl

import matplotlib.pyplot as plt

import os, gdown

os.makedirs('data/VHR10/', exist_ok=True)

url = 'https://drive.google.com/uc?id=1--foZ3dV5OCsqXQXT84UeKtrAqc5CkAE'
output_path = 'data/VHR10/NWPU VHR-10 dataset.rar'
gdown.download(url, output_path, quiet=False)

The code below downloads the VHR10 dataset to the data/VHR10/ location with the 'positive' split, and applies transformations using a given preprocessing function. The role of the preprocess function is to convert each image to a floating-point type and normalize the values to be between 0 and 1 by dividing by 255. This process helps prepare the data in a form suitable for model training.

def preprocess(sample):
sample["image"] = sample["image"].float() / 255.0
return sample

ds = VHR10(
root="data/VHR10/",
split="positive",
transforms=preprocess,
download=True,
checksum=True,
)

Exploring the VHR-10 Dataset

Let’s now take a look at the positive image set. There are a total of 650 images.

print(f"VHR-10 dataset: {len(ds)}")

Access the first item in the ds dataset and check the dimensions of the image corresponding to the "image" key. The shape attribute returns the dimensions of the image, which are represented in the format (number of channels, height, width).

ds[0]["image"].shape

torch.Size([3, 808, 958])

Let’s open the image of the sixth item at random. Change the dimensions of the image from (number of channels, height, width) to (height, width, number of channels) before displaying it.

image = ds[5]["image"].permute(1, 2, 0)
plt.imshow(image)
plt.show()

The VHR10 dataset in TorchGeo includes a plot method that visualizes annotations contained in annotations.json. These annotations provide information for object detection and instance segmentation, including visual elements like bounding boxes and masks. This allows for direct visualization of images with ground truth annotations marked on them.

ds.plot(ds[5])
plt.savefig('ground_truth.png', bbox_inches='tight')
plt.show()

Model Training

The following demonstrates the process of using PyTorch’s DataLoader to define a collate_fn function and utilize it to load data:

The collate_fn function extracts the image, boxes, labels, and masks from each item in a given batch and constructs a new batch dictionary. This newly constructed batch is then directly used for model training or evaluation. The DataLoader prepares the data in batch units for the model using this function. The option shuffle=True randomly shuffles the order of the dataset to reduce dependency on the sequence of data during model training.

def collate_fn(batch):
new_batch = {
"image": [item["image"] for item in batch], # Images
"boxes": [item["boxes"] for item in batch], # Bounding boxes
"labels": [item["labels"] for item in batch], # Labels
"masks": [item["masks"] for item in batch], # Masks
}
return new_batch # Return the new batch

Data Loader

dl = DataLoader(
ds, # Dataset
batch_size=32, # Number of data to load at one time
num_workers=2, # Number of processes to use for data loading
shuffle=True, # Whether to shuffle the dataset before loading
collate_fn=collate_fn, # collate_fn function for batch processing
)

This code defines a training class for object detection tasks and creates an instance of it. Importantly, the class is designed to handle variable-sized inputs.

The VariableSizeInputObjectDetectionTask class inherits from a standard ObjectDetectionTask and defines the training_step method to process variable-sized input images within each batch. This allows the model to effectively learn from input images of various sizes. The created instance is prepared to perform object detection tasks using the Faster R-CNN model with specified settings.

class VariableSizeInputObjectDetectionTask(ObjectDetectionTask):
# Define the training step
def training_step(self, batch, batch_idx, dataloader_idx=0):
x = batch["image"] # Image
batch_size = len(x) # Set batch size (number of images)
y = [
{"boxes": batch["boxes"][i], "labels": batch["labels"][i]}
for i in range(batch_size)
] # Extract bounding box and label information for each image
loss_dict = self(x, y) # Loss
train_loss: Tensor = sum(loss_dict.values()) # Training loss (sum of loss values)
self.log_dict(loss_dict) # Record loss values
return train_loss # Return training loss

task = VariableSizeInputObjectDetectionTask(
model="faster-rcnn", # Faster R-CNN model
backbone="resnet18", # ResNet18 neural network architecture
weights=True, # Use pretrained weights
in_channels=3, # Number of channels in the input image (RGB images)
num_classes=11, # Number of classes to classify (10 + background)
trainable_layers=3, # Number of trainable layers
lr=1e-3, # Learning rate
patience=10, # Set the number of patience iterations for early stopping
freeze_backbone=False, # Whether to train with the backbone network weights unfrozen
)
task.monitor = "loss_classifier" # Set the metric to monitor (here, the classifier's loss)

Prepare the configuration for model training using the PyTorch Lightning library. The settings below specify training the model with GPU, saving training logs and checkpoints to the ‘logs/’ directory, and setting the training to run for a minimum of 6 epochs and a maximum of 100 epochs.

trainer = pl.Trainer(
default_root_dir="logs/", # Set the default directory
accelerator="gpu", # Set the type of hardware accelerator for training (using GPU)
devices=[0], # List of device IDs to use ([0] means the first GPU)
min_epochs=6, # Set the minimum number of training epochs
max_epochs=100, # Set the maximum number of training epochs
log_every_n_steps=20, # Set how often to log after a number of steps
)

%%time

Model training

trainer.fit(task, train_dataloaders=dl)

Model Inference Example

Retrieve the next batch from the data loader (dl).

batch = next(iter(dl))

Obtain the model from the task (task) and set it to evaluation mode. This action deactivates specific layers, such as Dropout, that are used during training. Dropout is a highly effective regularization technique designed to prevent overfitting in neural networks.

Using torch.no_grad() disables gradient calculations, reducing memory usage and increasing computation speed. This is utilized when the model is not being updated during evaluation or inference phases. Now, pass the image batch through the model to obtain the prediction results.

model = task.model
model.eval()

with torch.no_grad():
out = model(batch["image"])

Define a sample for a specific batch index.

def create_sample(batch, out, batch_idx):
return {
"image": batch["image"][batch_idx], # Image
"boxes": batch["boxes"][batch_idx], # Actual bounding boxes
"labels": batch["labels"][batch_idx], # Actual labels
"masks": batch["masks"][batch_idx], # Actual masks
"prediction_labels": out[batch_idx]["labels"], # Labels predicted by the model
"prediction_boxes": out[batch_idx]["boxes"], # Bounding boxes predicted by the model
"prediction_scores": out[batch_idx]["scores"], # Confidence scores for each prediction
}

batch_idx = 0
sample = create_sample(batch, out, batch_idx)

Now, visualize the given sample. The plot method visualizes the image included in the sample, along with the actual labels and bounding boxes, and the predicted labels and bounding boxes.

ds.plot(sample)
plt.savefig('inference.png', bbox_inches='tight')
plt.show()

Visualizing Sample for Batch Index 3

batch_idx = 3
sample = create_sample(batch, out, batch_idx)

ds.plot(sample)
plt.show()

Visualizing Sample for Batch Index 5

batch_idx = 5
sample = create_sample(batch, out, batch_idx)

ds.plot(sample)
plt.show()

Isro free aiml courses

parmarjatin4911@gmail.com — Wed, 07 Aug 2024 18:21:03 +0000

https://elearning.iirs.gov.in/edusatregistration/

AI INSPIRATIONAL QUOTE

parmarjatin4911@gmail.com — Wed, 07 Aug 2024 18:11:25 +0000

Belief Creates the Fact