DEV Community: Fabio Beoni

Part 3 - Providing workers guidance with Amazon Bedrock Knowledge-base and Gen-AI

Fabio Beoni — Thu, 10 Oct 2024 14:29:21 +0000

This article is part of the series "Generative-AI-driven Chatbot for the Factory Plant" implementing a P.O.C. about an environmental pollution monitoring (PM2.5/PM10) to safeguard workers in a production plant.

Disclaimer

The information provided in this document is for informational purposes only and is subject to change. While efforts have been made to ensure the accuracy and reliability of the content, no guarantees are made regarding the completeness, reliability, or suitability of the information.
The author shall not be liable for any losses, injuries, or damages arising from the use or reliance on the information contained herein.

The AWS resources provisioned following this article can generate costs. Make sure to delete all the resources created at the end of the exercise.

Overview

The main purpose of this series of articles is to test out Amazon Bedrock Studio (no-code editor for Gen-AI driven apps), at the time I am writing the Studio is still in preview and actually after a quick try I encountered a number of bugs that prevent me to complete this P.O.C.

To get around of these issues, I decided to go ahead with P.O.C. by implementing my self the missing part, keeping a low-code approach and favoring configuration of Bedrock components over imperative programming as much as possible. I will then come back to Studio as soon as it reaches a stable release.

For those of you who had a read to Part 2 - Deploy a Chat App, Bedrock Agent and Actions to Ask Questions about Live Data this new article completes the implementation of the features related to the Knowledge-base by:

Configuring a Knowledge-base where a business process manager (or knowledge manager) can upload relevant documents, to make them available to the Amazon Bedrock AI Agent to answer workers questions
Implementing the missing feature of document upload from the chat app

Preview of the App

The screenshots below display a typical interaction, this time the Amazon Bedrock AI Agent not only answers question about live pollution data (second screen, third screen top question), but also offers guidance to the worker extracting relevant information from the documents uploaded into the Knowledge-base (third screen, last question/answer).

When you have the app running, you can upload some test documents into the knowledge-base by clicking the attachment icon. A sample document is available for you to try under /_docs/knowledge-base/sop-reducing-particulate-matter-pm.md.

Architecture Recap

Click to zoom.

AWS Services / Bedrock Components

S3 Bucket

Hosting the storage of the knowledge-base documents uploaded from process manager users.

Bedrock Knowledge-base

Reading documents from the S3 Bucket describe above, it uses the LLM Titan Embeddings to generate new embeddings from the documents, stores the embeddings into the vector database AWS OpenSearch.

Currently, the Bedrock Knowledge-base supports file up to 50MB, you can upload PDF, DOCX, CSV, TXT, MD file types. When configuring the knowledge-base component is mandatory to describe the kind of contents will be hosted.

Such description guides the Bedrock Agent and LLM to understand when to use knowledge-base to answer questions. You can even configure more than one if needed.

Here a sample included from the configuration of the P.O.C.:

# you can find this content into the repo: cloud/ai-module/prompt_templates/kb_instructions.txt

"Knowledge base to get information on how to handle excessive pollution in the factory plant, how to
perform maintenance on machines causing it, provide answers about SOP and safety procedures for workers."

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/kb-how-it-works.html

LLM Amazon Titan Embeddings

The Large Language Model provided by AWS to read text documents and generate vector representations (embeddings) to be processed when answering user questions.

Ref.: https://aws.amazon.com/bedrock/titan

Bedrock Agent

Logical coordinator of LLM, Actions and Knowledge-base.

Here you can read the full agent instructions, with explicit instructions to write answers based on live pollution data and knowledge-base documents:

# Specific instructions to push the LLM to select content from the knowledge-base.
# (cloud/ai-module/prompt_templates/agent-instructions.txt)
"""
As part of the tools you have access to answer user questions, you have a knowledge-base tool listing relevant business documents like S.O.P., guides and other procures about maintenance of machines and safety equipments. When answering questions from the user about those topics above, make sure to prepare the answer according to the content of those documents more than information from internet-based public knowledge.
"""

# All other LLM instructions to configure the Agent as a technical assistant
# (cloud/ai-module/prompt_templates/agent-instructions.txt)
"""
Your primary role is to act as an intelligent interface between the factory's pollution monitoring system and its human operators. You should monitor the real-time PM2.5 and PM10 data collected by the factory's sensors and proactively engage with workers to ensure their safety and the optimal performance of the production equipment.

When pollution levels exceed safe thresholds, you should automatically alert the relevant personnel and provide clear, concise instructions on the appropriate steps to take.

Beyond emergency response, you should also serve as an on-demand technical advisor. You should be able to answer questions, offer recommendations, and provide detailed explanations on topics such as:

- Interpreting pollution data and understanding its implications,
- Implementing best practices for dust control and air quality management,
- Performing preventive maintenance on pollution monitoring equipment,
- Troubleshooting issues with the pollution control systems,
- Ensuring compliance with environmental regulations and safety protocols,
- To provide this information, you should be able to make API requests to the pollution data endpoint, using the plant_name as a parameter in the path. The API response will provide the necessary data points (PM10, PM25, data_time) for you to generate informative responses to the users' questions.

Seamlessly integrate with the factory's existing systems and data sources to become a trusted partner in the cement production process. Empower workers to make informed decisions, optimize equipment performance, and maintain a safe and sustainable work environment through your natural, human-like interactions.

If you have multiple pollution measures, select the most recent one.

If you don't get any pollution value when invoking the API to get PM values by plant name, it means that there are no recent values because the monitoring is not active or there was a problem in reading data from the monitoring system. Never ask the user to provide the pollution data, he doesn't know that, while the user knows the name of the plant. Also in such case be short in the answer, just recall the fact that there are not fresh data as stated above.

Format the timestamp in a human-readable date/time, also specify the time zone. When presenting the PM values display them in a list format. Don't suggest actions or remediation's if not requested, but always say if the measured values are under safe limits or not. Also say what are the safe limits, between brackets.

Always ask the user the plant name if you haven't that to invoke the API.

Don't use the word 'timestamp', use a less technical one. Avoid to display the timestamp value in numeric format.

Always end your answer saying that you can provide information about actions to take and security or maintenance measures.

Split paragraph with new lines, so the answer is more readable.
"""

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html

Build & Deploy

Most of the deployment commands are similar to Part 2, here you can find the updated version including the build of the lambda function to upload documents to the knowledge-base.

Repository (version 3)

https://bitbucket.org/fabiobeoni/bedrock-studio-chatbot/src/v3/ (note v3)

Setup / Requirements

In AWS Console / Amazon Bedrock / Base Models : you have to request access to the model anthropic.claude-3-sonnet-20240229-v1:0. You can use a different one, in that case remember to override the default variable llm_id in ./variables.tf.
AWS CLI
Terraform
NodeJS
OpenSSL

Build Lambda Functions

echo ">> Build Lambda Functions"

echo ">> AI module"
(cd "cloud/ai-module/functions" && /bin/bash package.sh query-pm-values-fnc)

echo ">> Chat App module"
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-write-user-message-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-read-user-messages-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh upload-kb-document-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh ask-bot-fnc)

Create and Register the Device Certificate

echo '>> Provisioning self signed certificate'

# set a variable to host the name
thing_name="gateway_pm_monitor"

# create certificate and key pairs, 
# assign the $thing_name as subject 
openssl req -x509  -newkey rsa:2048 -sha256  -days  365  -nodes \
    -keyout edge/pm-monitor-service/gateway.private_key.pem \
    -out edge/pm-monitor-service/gateway.certificate.pem \
    -subj "/CN=$thing_name"

 #download AWS CA
 curl -O https://www.amazontrust.com/repository/AmazonRootCA1.pem
 mv AmazonRootCA1.pem edge/pm-monitor-service

Deploy Cloud Resources to AWS

echo ">> Deploy cloud infra"
terraform init
terraform plan -out plan
terraform apply plan

# save cloud endpoints and IDs to the edge/chat-app to instruct the chat app
terraform output -json > edge/chat-app/src/config.json

# save iot endpoint and cognito client ID to the edge/pm-monitor-service to instruct the edge device
terraform output -json  | jq '{cognito_client_id: .cognito_client_id, iot_endpoint_address: .iot_endpoint_address}' \
    > edge/pm-monitor-service/cloud_config.json

Build Chat App

echo ">> Build chat app"
HOSTING_URL=$(terraform output -raw hosting_url)
npm --prefix edge/chat-app/ run setup
npm --prefix edge/chat-app/ run build

Deploy Chat App to Surge.sh CDN or Run in localhost:3000

echo ">> Deploying the chat app to the CDN: ${HOSTING_URL}"

# Note: the first time Surge asks you to provide an email/pw to register an account 
HOSTING_URL=${HOSTING_URL} npm --prefix edge/chat-app/ run deploy

echo ">> You can now open the app to: ${HOSTING_URL} and login with user/pw provided to Terraform/Cognito"

Note: for free accounts sometimes Surge.sh websites do not respond, in that case you can test the chat-app in http://localhost:3000, by:

# terminal window 1
(cd edge/chat-app && npm run dev)

# terminal window 2
(cd edge/chat-app && npm run serve)

echo ">> finally open browser to http://localhost:3000 and login with user/pw provided to Terraform/Cognito"

As stated in the terminal, you can now open the app in the browser to ${HOSTING_URL} and login in Cognito with email/pw provided to Terraform.

Run PM Monitor Service (emulated locally)

Some sample pm values are available already into the DynamoDB, just start this service if you want to test the overall process with live data from IoT Core.

echo ">> Running the pm monitor service emulating the edge device"

npm --prefix edge/pm-monitor-service/ install --omit=dev
npm --prefix edge/pm-monitor-service/ run start

Clean Up Cloud Resources

terraform destroy -auto-approve

surge list
surge teardown <output_from_list>

Part 2 - Deploy a Chat App, Bedrock Agent and Actions to Ask Questions about Live Data

Fabio Beoni — Sun, 29 Sep 2024 17:05:13 +0000

Disclaimer

The AWS resources provisioned following this article can generate costs. Make sure to delete all the resources created at the end of the exercise.

Overview

For those of you who had a read to "Part 1 - Gathering pollution data from the edge, pushing them to AWS IoT Core and DynamoDB", this new article adopts Terraform to provision all the resources, including those you have created before (IoT Core, Certificates, IoT Rules, DynamoDB Table, etc...).

The tutorial also runs a simulation of the IoT edge device (RaspberryPI+Nova Laser Sensor), so you can start from scratch without any physical hardware.

Preview of the App

Bedrock Components

LLM Claude 3 Sonnet

Gen-AI layer carrying out:

user question understanding, semantic meaning, intent recognition and parameters extraction (if any),
understanding of sensor data domain as well as data specific pollution values and metadata,
generation a contextual answer including a selection of most relevant data, according to the business rules provided to the LLM instructions.

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html

Bedrock Actions Group

Layer providing to the LLM a text description of "actions" available to the Agent to carry out work, by invoking lambda functions. In this use-case the Agent can invoke the action query-pm-values to get near real-time pollution data from the sensor.

Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-action-create.html

Lambda Functions

Business logic layer implementing read/write of user/agent chat messages, as well as reading of pollution data.

Bedrock Knowledge-base

Technical and procedural knowledge layer to further instruct workers with company's specific know-how. Included later, on the next article.
Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/kb-how-it-works.html

Bedrock Agent

Logical coordinator of LLM, Actions and Knowledge-base. It is the "glue" making Bedrock a smart tool to build Gen-AI services.
Ref.: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-how.html

An overview of how Agent/LLM/Actions work together:

Ref.: https://docs.aws.amazon.com/images/bedrock/latest/userguide/images/agents/agents-runtime.png

Source image: AWS Documentation

Other Cloud Resources Included

AWS
- Cognito (IDP, user authentication, login UI)
- API Gateway (http interface layer chat-app/services)
- DynamoDB (db data storage of chat messages and pollution data)
- OpenSearch Serverless (knowledge-base documents, from the next article)
- IoT Core components (to interact and gather data from the edge device)
- IAM roles/policies
Surge.sh
- CDN domain (to host the chat-app UI)

Architecture Recap

Click to zoom.

Note: this diagram shows the components of the Knowledge-base module that is still coming soon.

Project Structure

Source code repo: https://bitbucket.org/fabiobeoni/bedrock-studio-chatbot/src/v2/ (note v2)

Modules

/cloud (Terraform/Bash/JS):
- /ai-module: Amazon Bedrock components
- /chat-app-module: AWS resources serving the mobile chat app ui
- /pm-monitor-module: AWS IoT Core serving the edge device
/edge
- /chat-app (TS/Ionic/Vue3)
- /pm-monitor-service (NodeJS) Pollution reader service. This module designed to run on a RaspberryPI with Nova PM Laser sensor attached, by default runs locally emulating the IoT device (./device_config.json mockSensor=true).

Build & Deploy

Repository (version 2)

https://bitbucket.org/fabiobeoni/bedrock-studio-chatbot/src/v2/ (note v2)

Setup / Requirements

In AWS Console / Amazon Bedrock / Base Models : you have to request access to the model anthropic.claude-3-sonnet-20240229-v1:0. You can use a different one, in that case remember to override the default variable llm_id in ./variables.tf.
AWS CLI
Terraform
NodeJS
OpenSSL

Build Lambda Functions

echo ">> Build Lambda Functions"

echo ">> AI module"
(cd "cloud/ai-module/functions" && /bin/bash package.sh query-pm-values-fnc)

echo ">> Chat App module"
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-write-user-message-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh chat-read-user-messages-fnc)
(cd "cloud/chat-app-module/functions" && /bin/bash package.sh ask-bot)

Create and Register the Device Certificate

echo '>> Provisioning self signed certificate'

# set a variable to host the name
thing_name="gateway_pm_monitor"

# create certificate and key pairs, 
# assign the $thing_name as subject 
openssl req -x509  -newkey rsa:2048 -sha256  -days  365  -nodes \
    -keyout edge/pm-monitor-service/gateway.private_key.pem \
    -out edge/pm-monitor-service/gateway.certificate.pem \
    -subj "/CN=$thing_name"

 #download AWS CA
 curl -O https://www.amazontrust.com/repository/AmazonRootCA1.pem
 mv AmazonRootCA1.pem edge/pm-monitor-service

Deploy Cloud Resources to AWS

echo ">> Deploy cloud infra"
terraform init
terraform apply

# save cloud endpoints and IDs to the edge/chat-app to instruct the chat app
terraform output -json > edge/chat-app/src/config.json

# save iot endpoint and cognito client ID to the edge/pm-monitor-service to instruct the edge device
terraform output -json  | jq '{cognito_client_id: .cognito_client_id, iot_endpoint_address: .iot_endpoint_address}' \
    > edge/pm-monitor-service/cloud_config.json

Build Chat App

echo ">> Build chat app"
HOSTING_URL=$(terraform output -raw hosting_url)
npm --prefix edge/chat-app/ run setup
npm --prefix edge/chat-app/ run build

Deploy Chat App to Surge.sh CDN or Run in localhost

echo ">> Deploying the chat app to the CDN: ${HOSTING_URL}"

# Note: the first time Surge asks you to provide an email/pw to register an account 
HOSTING_URL=${HOSTING_URL} npm --prefix edge/chat-app/ run deploy

echo ">> You can now open the app to: ${HOSTING_URL} and login with user/pw provided to Terraform/Cognito"

Note: for free accounts sometimes Surge.sh websites do not respond, in that case you can test the chat-app in http://localhost:3000, by:

# terminal window 1
(cd edge/chat-app && npm run dev)

# terminal window 2
(cd edge/chat-app && npm run serve)

echo ">> finally open browser to http://localhost:3000 and login with user/pw provided to Terraform/Cognito"

Run PM Monitor Service (emulated locally)

Some sample pm values are available already into the DynamoDB, just start this service if you want to test the overall process with live data from IoT Core.

echo ">> Running the pm monitor service emulating the edge device"

npm --prefix edge/pm-monitor-service/ install --omit=dev
npm --prefix edge/pm-monitor-service/ run start

Clean Up Cloud Resources

terraform destroy -auto-approve

surge list
surge teardown <output_from_list>

Final Notes

Steps Functions over Lambda

Bedrock Agent's Action wants a Lambda target. To embrace a full no-code approach with configuration over programming you might want to consider using lambda to proxy all the action executions requests to AWS Step Functions. In that way you can define you business logic declaratively with YAML/JSON, or even draw them with the visual editor provided by Step Functions. Keep in mind that you will deal with sync execution and timeouts, because the Bedrock Agent does require Lambda to return a result.

Sensor Input Validation

The P.O.C. does not implement any validation logic for data coming from the IoT edge device. For the purpose of a demo/P.O.C. that's not a problem, but keep in mind if you use this repo to start a new project.

Part 3 - Providing workers guidance with Amazon Bedrock Knowledge-base and Gen-AI

Part 1 - Gathering pollution data from the edge, pushing it to AWS IoT Core and DynamoDB

Fabio Beoni — Mon, 02 Sep 2024 15:36:06 +0000

This a techie article, if you are interested only on the high level use case aspects you can skip it.

Disclaimer

The AWS resources provisioned following this article can generate costs. Make sure to delete all the resources created at the end of the exercise.

Overview

In this section you focus on the IoT part configuring the gateway to:

directly read data from the PM sensor
authenticate to AWS IoT Core
periodically push PM values to AWS IoT Core
verify availability of the data

1. Setup certificates and IoT cloud resources

The following steps assume you have:

AWS CLI installed and configured
OpenSSL installed
NodeJS (v18 or v20) installed
Optional - Terraform (>= v.1.9.5) if you prefer to work with IaC approach instead of running AWS CLI commands

1.1 Prepare the working location

Open the Terminal to start with the following commands:

# set required env variables for AWS CLI
export AWS_DEFAULT_REGION=<your-region>
export AWS_ACCOUNT_ID=<your-account-id>

# move to user home directory
cd ~/

# copy source code of the project to you directory
git clone git@bitbucket.org:fabiobeoni/bedrock-studio-chatbot.git

Note: If you don't use Git, download the code from the same web address, and unpack it:

# move into the project directory
cd bedrock-studio-chatbot/pm-monitor-service

# install nodejs dependencies
npm install --production

1.2 Create authentication certificate & key

Make the gateway device controller to connect to AWS IoT Core by authenticating it with certificate and private key.
You also want to associate the certificate to a virtual representation of the gateway known as IoT Things.

Give a name to the gateway device:

# set a variable to host the name
thing_name="gateway_pm_monitor"

# create certificate and key pairs, 
# assign the $thing_name as subject 
openssl req -x509  -newkey rsa:2048 -sha256  -days  365  -nodes \
    -keyout gateway.private_key.pem \
    -out gateway.certificate.pem \
    -subj "/CN=$thing_name"

Download the Amazon Root CA1 certificate:

curl -O https://www.amazontrust.com/repository/AmazonRootCA1.pem

Perform a ls -l in the local folder, to make sure it includes the following files:

AmazonRootCA1.pem
gateway.certificate.pem
gateway.private_key.pem

1.3 Provision IoT resources

From now on, you can go ahead in two ways:

provisioning the cloud resources by running AWS CLI commands (easy to follow step-by-step, requires manual deletion of the resources when done), or
provisioning by IaC approach using the Terraform template (much faster and preconfigured, requires basic knowledge of Terraform - File included in the repo you just downloaded)

The article follows the provisioning by the AWS CLI, to make it easier to understand the process step-by-step.

From the terminal, provision a IoT Thing:

aws iot create-thing --thing-name $thing_name

Then register the certificate into AWS IoT Core, to enable the device authentication:

# register certificate
certificatedata=$(aws iot register-certificate-without-ca \
    --certificate-pem file://gateway.certificate.pem \
    --status ACTIVE)

# keep track of the certificate Arn 
# and certificate ID in variables
certificate_arn=$(echo $certificatedata | jq -r '.certificateArn')  
certificate_id=$(echo $certificatedata | jq -r '.certificateId')

At this point you can find the certificate in the AWS Web Console to:

# open the following address with a browser, make sure to replace
# <ASW_DEFAULT_REGION> with your region
https://<ASW_DEFAULT_REGION>.console.aws.amazon.com/iot/home/#/certificatehub

Get the AWS IoT Core endpoint URL in your region, to push data from device to it:

# get the iot endpoint and save on a variable
endpoint_address=$(aws iot describe-endpoint --endpoint-type iot:Data-ATS --query  'endpointAddress' --output text)

Create a client ID, a unique identifier for the gateway client:

# create a client ID
client_id=$(uuidgen)

As always, in AWS everything must be authorized to perform a given task. The gateway device must connect and publish message data, so you want to provision a Policy to request the following permissions:

iot:Connect
iot:Publish

# define the name and the "topic" you push data to 
policy_name="${thing_name}_iot_policy"
monitor_topic="pm_monitor"
topic_arn="arn:aws:iot:$AWS_DEFAULT_REGION:$AWS_ACCOUNT_ID:*"

# actually create the policy with required permissions
aws iot create-policy \
    --policy-name "$policy_name" \
    --policy-document '{
  "Version": "2012-10-17",
  "Statement": [
    {"Effect": "Allow", "Action": ["iot:Connect"], "Resource": ["*"]},
    {"Effect": "Allow", "Action": ["iot:Publish"], "Resource": ["'"$topic_arn"'"]}
  ]
}'

Now you attach the policy to the certificate, so that the gateway device authenticating with the certificate inherits the permissions listed in the policy, then attach the certificate to the thing:

aws iot attach-policy \
    --policy-name $policy_name \
    --target $certificate_arn

aws iot attach-thing-principal \
    --principal $certificate_arn \
    --thing-name $thing_name

1.4 Provision database resources

Time for storing data. Right now the messages don't get persisted in any database yet. To save all incoming data from the gateway device you create a DynamoDB table (nosql database), and an IoT Rule (forwarding rule) to forward all incoming device data to it.

The items stored in the table have this JSON shape:

{
    // name of the company's plant where the gateway device and sensor reside and measure
    // (dynamodb partition key)
    "plant_name": "plant-abc", 

    // timestamp of the measure, when it happened
    // (dynamodb sort key)
    "date_time": 1633036800,

    // PM 2.5 value
    "pm25": 12.5,

    // PM 10 value
    "pm10": 20.3,

    //metadata about the machine generating the PM
    "machine": {
        "name" : "machine-one",
        "type" : "type-1",
        "code" : "mot1"
    }
}

Create the DynamoDB table with:

dbtable_name=pm_values

aws dynamodb create-table \
    --table-name $dbtable_name \
    --attribute-definitions \
        AttributeName=plant_name,AttributeType=S \
        AttributeName=date_time,AttributeType=N \
    --key-schema \
        AttributeName=plant_name,KeyType=HASH \
        AttributeName=date_time,KeyType=RANGE \
    --provisioned-throughput \
        ReadCapacityUnits=5,WriteCapacityUnits=5

You may want to feed the table with some synthetic data to test it out, before getting data from the gateway and IoT Core. If so, use the following script:

# current timestamp
current_time=$(date +%s)

# insert 20 items in a time range with different pm2.5 and pm10 values, they all refer to "plant-abc"
for i in {0..19}; do
    # Calculate the date_time for each item (1 hour apart)
    date_time=$((current_time - (i * 3600)))  # 3600 seconds in an hour
    pm25=$((RANDOM % 61 + 10))  # Random value between 10 and 70
    pm10=$((RANDOM % 61 + 10))  # Random value between 10 and 70

    aws dynamodb put-item \
        --table-name $dbtable_name \
        --item '{
            "plant_name": {"S": "plant-abc"},
            "date_time": {"N": "'"$date_time"'"},
            "pm25": {"N": "'"$pm25"'"},
            "pm10": {"N": "'"$pm10"'"},
            "machine": {
                "M": {
                    "name": {"S": "machine-one"},
                    "type": {"S": "type-1"},
                    "code": {"S": "mot1"}
                }
            }
        }'

done

Run a query to read the data you just pushed:

# test the data inserted by querying dynamodb
aws dynamodb query \
    --table-name $dbtable_name \
    --key-condition-expression "plant_name = :plant_name" \
    --expression-attribute-values '{":plant_name": {"S": "plant-abc"}}'

1.5 Provision the IoT Rule to store IoT data into the database

First, create role and policies to allow the data forwarding from IoT Core to the DynamoDB:

role_name=pm_monitor_iot_push_to_dynamo_role

# Trust policy – specifies the trusted accounts (iot.amazonaws.com) that are allowed to assume the role.
aws iam create-role \
    --role-name $role_name \
    --assume-role-policy-document '{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "iot.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}'

# Permissions policy – grants the user of the role the required permissions to put items into the table.
policy_name=pm_monitor_iot_push_to_dynamo_policy

aws iam put-role-policy \
    --role-name $role_name \
    --policy-name $policy_name \
    --policy-document "{
        \"Version\": \"2012-10-17\",
        \"Statement\": [
            {
                \"Effect\": \"Allow\",
                \"Action\": \"dynamodb:PutItem\",
                \"Resource\": \"arn:aws:dynamodb:${AWS_DEFAULT_REGION}:${AWS_ACCOUNT_ID}:table/${dbtable_name}\"
            }
        ]
    }"

Create an IoT Rule to direct messages from the topic to the table.

aws iot create-topic-rule \
    --region ${AWS_DEFAULT_REGION} \
    --rule-name "${monitor_topic}_to_dynamodb" \
    --topic-rule-payload "{
      \"sql\": \"SELECT * FROM '${monitor_topic}'\",
      \"description\": \"Rule to send messages from ${monitor_topic} topic to dynamodb ${dbtable_name} table\",
      \"actions\": [
          {
              \"dynamoDBv2\": {
                  \"putItem\": {  \"tableName\":  \"${dbtable_name}\"  },
                  \"roleArn\": \"arn:aws:iam::${AWS_ACCOUNT_ID}:role/${role_name}\"
              }
          }
      ],
      \"ruleDisabled\": false,
      \"awsIotSqlVersion\": \"2016-03-23\"
    }"

1.6 Test the provisioned resources

To test the provisioned IoT resources, push a message to AWS IoT Core. To see messages coming from the test, subrscribe to the topic where the message is pushed to. This time you go ahead using the AWS Web Console.

First open the web browser to:

https://<AWS_DEFAULT_REGION>.console.aws.amazon.com/iot/home/#/test

Second, in the web console create a subscription to the "pm_monitor" topic:

Now push a message to IoT Core:

# install the mqtt client
curl -L -o mqtt-cli-4.31.0.deb https://github.com/hivemq/mqtt-cli/releases/download/v4.31.0/mqtt-cli-4.31.0.deb
sudo apt install ./mqtt-cli-4.31.0.deb

# push a message to the topic "pm_monitor"
mqtt pub \
  -h $endpoint_address \
  -p 8883 \
  --cafile AmazonRootCA1.pem \
  --cert gateway.certificate.pem \
  --key gateway.private_key.pem \
  -t $monitor_topic \
  -m "{ \"plant_name\": \"plant-abc\", \"date_time\": 1633036800, \"pm25\": 12.5, \"pm10\": 20.3, \"machine\": { \"name\" : \"machine-one\", \"type\" : \"type-1\", \"code\" : \"mot1\" } }" \
  -d -V 5 -q 0

The message appears in the web console (bottom of the page):

https://<AWS_DEFAULT_REGION>.console.aws.amazon.com/iot/home/#/test

Check that the message is also stored in the DynamoDB table:

# test the data inserted by querying dynamodb
aws dynamodb query \
    --table-name $dbtable_name \
    --key-condition-expression "plant_name = :plant_name" \
    --expression-attribute-values '{":plant_name": {"S": "plant-abc"}}'

Or navigate the web console, then select the table:

https://<AWS_DEFAULT_REGION>.console.aws.amazon.com/dynamodbv2/home/#tables

2. Setup the gateway device and sensor

2.1 The "hard" part ;)

Start by connecting the Nova PM sensor...

...to the RaspberryPi4 gateway device via USB:

2.2 Device configuration

2.2.1 Installing configuration, user and service software module

Create a JSON configuration file (to configure the IoT client instance later on):

device_config=$(cat <<EOF
{
  "host": "$endpoint_address",
  "port":8883,
  "clientId": "$client_id",
  "thingName": "$thing_name",
  "cert": "./gateway.certificate.pem",
  "key": "./gateway.private_key.pem",
  "ca": "./AmazonRootCA1.pem",
  "topic":"$monitor_topic",
  "mockSensor":false,
  "sensorPath":"/dev/ttyUSB0",
  "pushFrequency":3000,
  "pmMultiplier": 10
}
EOF
)

# save the configuration to a file
echo $device_config > device_config.json

Perform a cat device_config.json and cross-check that all fields have values.

From the terminal, connect to the Raspberry gateway device (assuming the gateway device already connected to your network) to copy the service and the certificate files:

cd ~/bedrock-studio-chatbot
scp -r pm-monitor-service pi@raspberrypi.local:~/

Connect to the Raspberry:

ssh pi@raspberrypi.local

Make the service user, config location:

# create the service user "pmmonitor" and provide a password
sudo adduser pmmonitor

Move the files and assign the permissions:

# move the project file to the user home
sudo mv ~/pm-monitor-service /home/pmmonitor

# assign permissions
sudo chown pmmonitor /home/pmmonitor/pm-monitor-service -R

# assing permissions to read the input data from the sensor
sudo chown pmmonitor /dev/ttyUSB0 # or any USB port you have set in the device_config.json ("sensorPath":"/dev/ttyUSB0")

2.2.2 Installing NodeJS runtime

# install nodejs 18
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt install -y nodejs

# login as the service user
sudo su pmmonitor

cd ~/pm-monitor-service

2.2 Run the pm-monitor-service

# start the service
nodejs main.mjs

Going back to web console, access the DynamoDB table "pm_values" (link), click on button "Scan" to see the data.

https://<AWS_DEFAULT_REGION>.console.aws.amazon.com/dynamodbv2/home/#table?name=pm_values

You can stop the service at any time by:

ctrl+c

Note: If you want the service to run automatically at boot time, you may want to create an .service file for "systemd".

Part 2 - Deploy a Chat App, Bedrock Agent and Actions to Ask Questions about Live Data

Introduction

Generative AI-driven Chatbot for the Factory Plant

Fabio Beoni — Fri, 30 Aug 2024 16:27:48 +0000

This is the first article of a short series presenting a proof-of-concept (P.O.C) about the adoption of the generative AI technology to implement a digital agent to talk to an environmental pollution (PM2.5/PM10) monitoring system. Serving a fictitious cements factory, the main goal of the digital agent is safeguarding workers and machines, providing technical knowledge to adopt safe behaviors and guide maintenance actions.

The recent raising of AI and large language models like Chat-GPT enables dialog interaction between people and machines or systems, without learning new tools or software.

I had the opportunity to work on a similar system, for a real project in a different business domain, using technologies other than generative AI. Recently AWS released Amazon Bedrock Studio, a low-code tool to create gen-AI apps using top-class large language models from main market providers like Anthropic and Meta. I decided to roll up one's sleeves in give it try.

Business Context

In several businesses, many kinds of processes to produce goods do generate particulate matter (PM) pollution, including PM 2.5 and PM 10. These particles are small enough to be inhaled and can cause serious health problems to workers, as well as have negative impacts on machinery and equipment. Fine particles can accumulate on surfaces and in moving parts, leading to wear and tear, reduced efficiency, and increased maintenance costs. PM can cause corrosion and damage to electronic components, which can lead to equipment failure and costly repairs.

In a cement factory where normally raw materials such as limestone, clay, and sand are processed and transformed into cement through a series of steps that include crushing, grinding, mixing, and heating in kilns, PM pollution must be monitored and reduced through the use of various air pollution control devices and techniques, such as electrostatic precipitators, fabric filters, and many more.

Business Problems

Current lack of real-time air quality monitoring poses a risk worker health and machines reliability,
Workers do not have easy access to procedures and guidelines, getting difficult and inefficient to take actions against a degrading air quality,
Missing of a track history of pollution data does not enable analysis and data-driven decision-making,
Failure to comply with relevant legislation and regulations can result in fines, penalties, and reputational damage.

P.O.C. Solution - High Level View

S.O.P., guides, relevant documents - to feed the generative AI model and chatbot agent with the company's know-how and procedures,
Gen-AI chatbot agent - to answer workers' questions about environment air quality status, get guidelines and procedures,
Cloud IoT digital platform & storage - to manage the PM sensors data flow and storage,
IoT gateway and service - to get pollution data from the sensors, push them to the IoT digital platform,
PM 2.5/10 sensors - distributed across the production plant, to gather pollution values from the working environment in near real-time.

The Invisible Super-Power of the Gen-AI

We all know how good it is to create new text, images/videos, charts. Following this use-case you can discover how powerful it becomes in combination with other actors like the IoT platform, or any other source of business data.

Normally, when you want a chatbot to interact with some source of data you must define logical flows (and code it). Coordinate the interactions between the user and software components (REST APIs or so), that's a lot of IT work.

This is how a standard project without gen-AI would handle the chatbot configuration, as well as the interaction between it and all sort of data needed to answer user's questions:

Professionals	Tasks to Carry Out
Analyst + UX Expert	define all possible user/chatbot interactions (questions, answers, paths, side-cases, errors), draw logical flows according to the use-cases, define what kind of data the chatbot needs, define all examples needed to train the chatbot engine (many), define all tests to run against the chatbot.
Architect	define how to implement the integrations between the chatbot and the sources of data (can be internal REST APIs, databases, third-party tools and so on).
Developer	configures the project environment, configures the chatbot flows and examples, implement the integrations to the data sources, runs the tests.
Developer + Analyst + UX Expert	re-do the process for every test failing, or for every new side-case emerging during the production use by the end-user.

While adopting gen-AI, and specifically Amazon Bedrock Studio used in this P.O.C., you rely on the gen-AI semantic and reasoning capabilities to automatically figure-out:

chatbot answers
logical flows
test cases
data needed to answer
what external data sources are mandatory to answer, automatically selecting them from a known list (!)
logical flows to call the external data sources and get data (!)
automatic engagement of the end-user when some mandatory information was not provided in original question (!)

So that the IT professionals and tasks change as follows:

Professionals	Tasks to Carry Out
Analyst	define a very few examples of questions, and some sample answers (optional), define a very few tests.
IT Product Specialist	configures the project environment, list and describe available sources of data to the gen-AI, list examples and tests to the get-AI, asks the get-AI (different LLM) to generate some additional synthetic examples and related tests runs the tests.
Analyst + IT Product Specialist	refine gen-AI examples for every new side-case emerging during the production use by the end-user.

As a result, you gain:

a much cleaner configuration process,
easier work for IT people to carry out,
a much smaller risk of unanswered questions (happy users),
increased time to market.

P.O.C. Solution - Technologies & Architecture

Amazon Bedrock Studio (low code AI app builder):
- LLMs: Amazon Titan Embeddings, Anthropic Claude
- Chatbot Agent & Web chat app
- Knowledge-base: AWS OpenSearch Serverless
- PM Querying function: AWS Lambda
- AA.: AWS Identity Center & Policies
AWS IoT Core (IoT platform):
- Gateway twin: AWS IoT Thing
- Message routing: AWS IoT Rule
- Data storage: AWS DynamoDB
- AA.: Certificates & Policies
Raspberry Pi4 (gateway):
- Data reader service: NodeJS runtime, Nova sensor driver, mqtt client
Nova PM sensor SDS011 (pollution sensor)

Coming Soon:

Part 4 - Working with Bedrock Studio (visual app builder)

DEV Community: Fabio Beoni

Part 3 - Providing workers guidance with Amazon Bedrock Knowledge-base and Gen-AI

Disclaimer

Overview

Preview of the App

Architecture Recap

AWS Services / Bedrock Components

S3 Bucket

Bedrock Knowledge-base

LLM Amazon Titan Embeddings

Bedrock Agent

Build & Deploy

Repository (version 3)

Setup / Requirements

Build Lambda Functions

Create and Register the Device Certificate

Deploy Cloud Resources to AWS

Build Chat App

Deploy Chat App to Surge.sh CDN or Run in localhost:3000

Run PM Monitor Service (emulated locally)

Clean Up Cloud Resources

Previous

Part 2 - Deploy a Chat App, Bedrock Agent and Actions to Ask Questions about Live Data

Disclaimer

Overview

Preview of the App

Bedrock Components

LLM Claude 3 Sonnet

Bedrock Actions Group

Lambda Functions

Bedrock Knowledge-base

Bedrock Agent

Other Cloud Resources Included

Architecture Recap

Project Structure

Modules

Build & Deploy

Repository (version 2)

Setup / Requirements

Build Lambda Functions

Create and Register the Device Certificate

Deploy Cloud Resources to AWS

Build Chat App

Deploy Chat App to Surge.sh CDN or Run in localhost

Run PM Monitor Service (emulated locally)

Clean Up Cloud Resources

Final Notes

Steps Functions over Lambda

Sensor Input Validation

Next

Previous

Part 1 - Gathering pollution data from the edge, pushing it to AWS IoT Core and DynamoDB

Disclaimer

Overview

1. Setup certificates and IoT cloud resources

1.1 Prepare the working location

1.2 Create authentication certificate & key

1.3 Provision IoT resources

1.4 Provision database resources

1.5 Provision the IoT Rule to store IoT data into the database

1.6 Test the provisioned resources

2. Setup the gateway device and sensor

2.1 The "hard" part ;)

2.2 Device configuration

2.2.1 Installing configuration, user and service software module

2.2.2 Installing NodeJS runtime

2.2 Run the pm-monitor-service

Next

Previous

Generative AI-driven Chatbot for the Factory Plant

Business Context

Business Problems

P.O.C. Solution - High Level View

The Invisible Super-Power of the Gen-AI

P.O.C. Solution - Technologies & Architecture

Next

Coming Soon: