DEV Community: Wendy Wong

Getting started with Amazon Nova 2 Omni (Preview) & Amazon Nova AI hackathon

Wendy Wong — Fri, 20 Feb 2026 03:50:25 +0000

What's hot off the press from AWS re:Invent 2025?

I am still excited to experiment with AI foundation models announced at last year's AWS re:Invent. If you missed watching the keynote from CEO of Amazon Web Services Matt Garman you may keep abreast of the key announcements if you are practitioner or researcher working at the intersection of data and AI.

Let me introduce you to Amazon Nova 2 family of foundation models which are foundation models that include different capabilities to help you achieve your tasks.

You may refer to the technical report published in December 2025 by Amazon Artificial Generative Intelligence team to compared benchmark performance.

Amazon Nova 2 Lite: a cost-effective multimodal reasoning model that is fast and output text and code.
Amazon Nova 2 Pro: a multimodal model that can be configured for extended thinking with multimodal reasoning capabilities.
Amazon Nova 2 Sonic: a speech-to-speech model that can be used for conversational AI such as building chatbots and in contact centers to design customer experiences.
Amazon 2 Omni (preview): a one-size-fits-all multimodal reasoning model that can handle inputs(up to 1 million tokens) from text, images, video or speech and generates text and image outputs.

Lesson Objectives

In this lesson you will learn the following:

What is Amazon Nova 2 Omni (Preview)
What are the features?
What are the benefits?
How do I get started?

What is Amazon Nova 2 Omni (Preview)

Let me introduce you to Amazon 2 Omni which is a unified multimodal reasoning model that was launched on 3 December 2025 at AWS re:Invent and is currently in public preview.

It can be used for reasoning and image generation.

What are the features?

Amazon Nova 2 Omni includes the following features:

Reduces costs
Text input: available in 200 languages
Speech input: available in 10 languages
A context window of 1M tokens

What are the benefits?

Amazon Nova 2 Omni reduces the complexity of managing several AI models.

How do I get started?

In public preview, you may access Amazon Nova 2 Omni if you are an Amazon Nova Forge customer. You can speak to your AWS account team.

AWS re:Invent 2025

If you missed last year's AWS re:Invent you may watch the following video with technical demonstrations from the session:

AWS re:Invent 2025 - [NEW LAUNCH] Amazon Nova 2 Omni: A new frontier in multimodal AI (AIM3324).

Further Resources

If you would like to learn more about Amazon Nova 2 Omni in Preview, you may browse the technica report card and also the developer guide below:

Amazon Nova 2: Multimodal reasoning and generation models
Amazon Nova Developer Guide for Nova 2

This Month - February 2026

If you would like to get involved and build your cloud skills you may join the AWS Community to:

AWS Builder Center: Learn and build solutions in a vibrant community of builders.
AWS re:Invent 2025 reCap: locate and join an event in your city.
AWS re:Invent 2025 - all the keynotes, innovation talks and break out sessions are listed under AWS events.
Stay connected to the AWS Community by joining a User Group in your city or if you are a university student you may join a Cloud Club near you - the local communities promote diversity, inclusion and also help you to upskill with inspiring technical speakers.

Amazon Nova AI Hackathon - 2 February to 16 March, 2026

You are invited to participate in the Amazon Nova AI hackathon, share your journey of building an innovative solution and you may win AWS credits worth $200.

You may choose from the following categories to innovate with Amazon Nova such as:

Agentic AI
Multimodal understanding
UI automation
Voice experiences

You may find more details from this link and how you may participate.

Amazon Nova AI Hackathon | AWS Developers

Participating in the Amazon Nova AI Hackathon? Did you know if you share your journey in a blog, you can earn $200 AWS Credits?! AWS is awarding prizes to developers who share their hackathon stories on AWS Builder Center. What makes a good submission: → Your project impacts the target community → It has real-world applications and measurable benefits → There are plans for encouraging adoption Hackathon ends March 16, 2026. Share your journey: https://go.aws/46Knu4i

linkedin.com

You are welcome to subscribe to my monthly newsletter Coffee n Learn, Becoming: Women in Engineering or take one of my LinkedIn Learning courses to upskill in data analytics or generative AI.

Until the next update happy learning! 😁

Build AI agents and automate (UI) workflows from your web browser - Amazon Nova Act and Kiro

Wendy Wong — Tue, 30 Dec 2025 06:42:27 +0000

Reimagine the possible with AI agents

Building workflows takes time and effort to maintain for developers and engineers. With designing automated workflows we need to consider our business processes and logic. At AWS re:Invent 2025, on Day 3, AWS VP of Agentic AI Dr Swami Sivasubramanian announced the general availability of Amazon Nova Act in the AWS region United States (N.Virginia) us-east-1.

Lesson Objectives

In this lesson, you will learn how to:

Build an AI agent from UI automated workflows
Identify use cases for Amazon Nova Act

What is Amazon Nova?

AWS introduces a family of AI offerings under Amazon Nova which include the following:

a) Amazon Nova 2 Foundation Models:

Nova 2 Lite - This is foundation model cost effective for reasoning tasks such as building chatbots. The data type includes: text, video and image.
Nova 2 Pro (preview) - This is an intelligent reasoning model suitable for AI tasks such as agentic coding. The data type includes: text, video,image and speech.
Nova 2 Omni (preview) - This foundation model is capable of
Nova 2 Sonic - This is a speech-to-speech model that can be used for conversational AI. The data type includes: text and speech.
Nova multimodal embeddings - This is a multimodal embedding model that can be used for agentic RAG and semantic search. The data type includes: text, image, audio and video.

b) Frontier Model:

Amazon Nova Forge - This frontier model allows you to combine your organizations' data with curated training data from Amazon Nova.

c) Automate web browser workflows at scale

Amazon Nova Act - This AWS service that built on the architecture of Nova 2 Lite to create AI agents to automate web browser UI workflows.

What is Amazon Nova Act?

Amazon Nova Act is an AWS service that helps you to build and manage AI agents tasked with automating UI workflows at massive scale for repetitive tasks.

Take a closer look at how Amazon Nova Act is used at Amazon AGI Lab and how it become generally available for web browser automation to assist with taking action and making decisions.

What are the features?

With web browser automated workflows the capabilities of Amazon Nova Act include:

Extracting information
Confirming bookings
Filling out forms
Perform a web search

Agents may be quickly created with natural language prompts and python code.

What are the benefits?

Amazon Nova act completes repetitive tasks, launches APIs and can also perform escalations.
Develop UI automation agents from prototype to production in a matter of hours.
Deploy and manage fleets of agents

What are the use cases?

The critical UI workflow automation from your browser can solve tasks such as:

Data entry - Use agents to help you populate details and remove the need to copy and paste data.
Web QA testing - validating user journey workflow in a web browser instead of using test scripts.
Checkout flows - the agent may take actions to complete a workflow for example a purchase, refund, validation and UI changes.
Data extraction - navigate websites and extract data from unstructured data sources that cannot be programmed or easily exported.

Task

Build an AI agent to assist a first-year psychology student enrol into an Australian university

Tutorial 1: Build reliable AI agents for UI workflow automation with Amazon Nova Act and Kiro

Step 1: Launch Kiro (i.e. agentic IDE) from your desktop and sign in with your AWS Builders ID.

Allow Kiro to access your data. Select Allow access.

Step 2: In Kiro, navigate on the left-handside to Extension and search for Amazon Nova Act to install the extension.

Select Trust publisher and install to install the extension.

Step 3: Now start developing a prototype to build an AI agent with Amazon Nova Act for automating UI workflow from a web browser.

You may experiment with your user journeys in the Amazon Nova Act Playground and generate an API key from the link:

https://nova.amazon.com/act

I am not able to access this playground from my location in Sydney, Australia (ap-southeast-2).

Step 4: Create your first workflow

With the Nova Act extension installed, in the Kiro terminal enter:

Upgrade to the latest version

pip install --upgrade nova-act

Step 5: Create a project folder a name it 'UTS enrolment folder' and save it in your working directory such as Documents

Step 6: In Kiro IDE go to File -> Open Folder -> UTS enrolment folder

In Kiro select vibe coding and develop a prototype in this agentic IDE and enter this prompt:

Kiro will create a prototype and save the files in your project folder.

This is a screenshot of the html file for the workflow:

You can open the 'uts-psychology-enrollment-workflow' html file and have a look at the prototype and share with your team.

You may also inspect the README file.

Step 4: You may create the Steering files using Kiro.

Step 5: You may also create your SPECS using Kiro.

With a staged process, Kiro will design the:

a) Requirements

b) Design document

c) Analyze the acceptance criteria

d) Design Implementation Plan

If you inspect your project folder, you will be able to see a subfolder titled '.kiro' automated and it includes both the steering files and the specs.

Kiro has completed all the specifications for the development from the beginning.

Tutorial 2: Deploy the web browser workflow with Amazon Q extension in Visual Studio

Step 1: Open Visual Studio Code and Open a Folder 'UTS enrolment folder'.
Step 2: Open chat in Visual Studio and enter the prompt.

Step 3: In Visual Studio with Amazon Q we can deploy the workflow into AWS via our IAM credentials from the specs created in Kiro.
Two deployment files were created:
Deployment script

# UTS Psychology Enrollment - AWS Deployment Script
param(
    [string]$BucketName = "uts-psychology-enrollment",
    [string]$Region = "us-east-1"
)

Write-Host "=== UTS Psychology Enrollment - AWS S3 Deployment ===" -ForegroundColor Cyan
Write-Host ""

# Check AWS CLI
Write-Host "Checking AWS CLI configuration..." -ForegroundColor Yellow
aws sts get-caller-identity 2>&1 | Out-Null
if ($LASTEXITCODE -ne 0) {
    Write-Host "ERROR: AWS CLI not configured. Run: aws configure" -ForegroundColor Red
    exit 1
}
Write-Host "✓ AWS CLI configured" -ForegroundColor Green

# Create bucket
Write-Host ""
Write-Host "Creating S3 bucket: $BucketName..." -ForegroundColor Yellow
aws s3 ls "s3://$BucketName" 2>&1 | Out-Null
if ($LASTEXITCODE -eq 0) {
    Write-Host "✓ Bucket already exists" -ForegroundColor Green
} else {
    aws s3 mb "s3://$BucketName" --region $Region 2>&1 | Out-Null
    if ($LASTEXITCODE -eq 0) {
        Write-Host "✓ Bucket created" -ForegroundColor Green
    } else {
        Write-Host "ERROR: Failed to create bucket" -ForegroundColor Red
        exit 1
    }
}

# Enable website hosting
Write-Host ""
Write-Host "Enabling static website hosting..." -ForegroundColor Yellow
aws s3 website "s3://$BucketName" --index-document uts-psychology-enrollment-workflow.html
Write-Host "✓ Website hosting enabled" -ForegroundColor Green

# Set up public access
Write-Host ""
Write-Host "Configuring public access..." -ForegroundColor Yellow

$policy = @"
{
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "PublicReadGetObject",
        "Effect": "Allow",
        "Principal": "*",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::$BucketName/*"
    }]
}
"@

$policy | Out-File -FilePath "bucket-policy.json" -Encoding utf8

aws s3api put-public-access-block --bucket $BucketName --public-access-block-configuration "BlockPublicAcls=false,IgnorePublicAcls=false,BlockPublicPolicy=false,RestrictPublicBuckets=false" 2>&1 | Out-Null
aws s3api put-bucket-policy --bucket $BucketName --policy file://bucket-policy.json 2>&1 | Out-Null
Write-Host "✓ Public access configured" -ForegroundColor Green

# Upload file
Write-Host ""
Write-Host "Uploading HTML file..." -ForegroundColor Yellow
aws s3 cp uts-psychology-enrollment-workflow.html "s3://$BucketName/" --content-type "text/html"
if ($LASTEXITCODE -eq 0) {
    Write-Host "✓ File uploaded successfully" -ForegroundColor Green
} else {
    Write-Host "ERROR: Upload failed" -ForegroundColor Red
    exit 1
}

Remove-Item "bucket-policy.json" -ErrorAction SilentlyContinue

# Show results
Write-Host ""
Write-Host "=== Deployment Complete ===" -ForegroundColor Green
Write-Host ""
Write-Host "Website URL:" -ForegroundColor Cyan
Write-Host "  http://$BucketName.s3-website-$Region.amazonaws.com" -ForegroundColor White
Write-Host ""

A deployment.md file was created

You may also deploy this workflow on Amazon Nova Act CLI and you may productionize this workflow on Nova Act CDK templates.

More learning resources

AWS Community Builders Program - January 2026

Are your a community builder? The annual application intake for the AWS Community Builders program is opening in early January.

This is your chance to shine if you enjoy helping people to build their knowledge through meetups, public speaking at conferences, writing blogs and creating technical content such as videos or podcasts.

You may place your name on the waitlist to be notified when the AWS Community Builders program opens!

Until the next update, happy learning and happy new year! 😀

Getting started in AWS Clean Rooms with privacy-enhanced synthetic data generation

Wendy Wong — Sun, 28 Dec 2025 06:19:10 +0000

What is hot off the press ✨ from AWS re:Invent 2025 for Analytics?

I am still enthralled and excited about the data and AI innovation for data scientists, researchers, data engineers and data analysts from keynotes from Amazon Web Services CEO Matt Garman on Day 2 and VP of Agentic AI Dr Swami Sivasubramanian on Day 3 in their announcements.

Maintaining data privacy is important for enterprises and data science teams involved in training machine learning models.

Let me introduce you to a new feature for AWS Clean Rooms to enhance privacy in synthetic data generation for machine learning model training.

Lesson Objectives

In this lesson you will learn the following:

What are AWS Clean Rooms?
What are the benefits?
What are the new features?
What are the use cases?
How do I get started?

What are AWS Clean Rooms?

AWS Clean Rooms is an AWS service that allows organizations to analyze their data in a secure environment and also collaborate with others without sharing their underlying proprietary data.

What are the benefits?

AWS Clean Rooms allows organizations to generate data insights from multiple companies without having to physically move raw data. You may use APIs to include AWS Clean Rooms in your company's workflow.
For example, if you work on a multi-agency government transformation project, you may create permissions to your raw data in AWS or Snowflake environment and allow contractors or another team's data analysts access to collaborate with system analysts from the transformation department using zero-ETL.

You do not need a data engineer to build or maintain a complex data pipeline once you add permissions in AWS Clean Rooms, other participants may collaborate and access data from another company or department.

Link and match customer recordsfrom multiple companies to train and deploy machine learning models. You may even bring in your own machine learning model and deploy it to access data insights from other companies without sharing a custom ML model or raw data.

You may also integrate other AWS services with AWS Clean Rooms such as Amazon Athena, AWS Glue, Amazon S3, AWS Secrets Manager, AWS CloudTrail and CloudFormation.

What are the new features?

At AWS re:Invent 2025, AWS Clean Rooms launched a new feature to enhance the privacy of generating synthetic data for a custom ML model which became generally available on November 30 2025.

This new feature allows teams to create a synthetic version of sensitive data that may be used in a secure environment to train machine learning models.

With synthetic data generation, people and entities from the original dataset will be de-identified.

What are the use cases?

With privacy-enhancing synthetic data generation there are more use cases for machine learning industries such as healthcare, government, defence, marketing and more.

This allows data scientists and analysts to access granular data for machine learning model training.

Considerations before you get started with synthetic data generation:

You need to configure details of who will pay for the synthetic dataset
Synthetic data generation does not remove or redact sensitive values (i.e. PII) from the original dataset.
Does not support data generation from text data.

How do I get started?

There are pre-requisites for establishing an AWS Clean Room which include:

Sign up for an AWS account
Set up service roles for AWS Clean Rooms such as administrator or collaboration member
Set up service roles for AWS Clean Rooms ML

Tutorial: Synthetic dataset generation for ML model training

Step 0: Login into your AWS account as an IAM Admin User. (Note: If you do not have an AWS account you can create one here).

Navigate to AWS Clean Rooms and today I am working from the AWS Sydney region (ap-southeast-2). This is the list of other AWS supportedregions for AWS Clean Rooms.

Step 1: A collaboration member creates an analysis template that includes:

a) SQL defines the dataset.

b) Privacy-related configurations ensure the synthetic data meets data providers’ compliance requirements.

Select Create Collaboration

Define the Collaboration by entering a name and a description followed by member details.

Specify member abilities

a) Identify which user will run SQL queries
b) Identify who will receive the results of the analysis
c) Identify which user will train the machine learning models
d) Identify who receive the output from inference

Configure payment

For building analysis queries and also machine learning model purpose-built workflows you may choose who is responsible for the payment.

Configure Membership

You may specify which Amazon S3 bucket will include your machine learning model output and the result format of the analysis e.g. csv file or parquet.

Before creating membership, you will also need to check the box that you agree to pay for the compute costs of collaboration.

Step 2: Data providers approve the require analysis template, the collaboration query creates a machine learning input channel.

You may classify the columns in your data output schema as categorical or numerical.

Step 3: Clean Rooms ML generates a synthetic dataset and verifies it meets privacy thresholds in the analysis template.

You may adjust the privacy threshold in % between the range of 50 to 100 to ensure that member of the raw data cannot be identified.

Step 4: Once thresholds are satisfied, the ML input channel populates the synthetic dataset.

You may adjust the privacy level in epsilon to match your organizations' privacy requirements.

Step 5: Customers use the ML input channel to train the custom ML model linked to the collaboration.

More Learning Resources

AWS Clean Rooms User Guide
AWS On Air- Collaborate across multiple data sources and clouds in AWS Clean Rooms
Creating a configured table with Snowflake data source
AWS Clean Rooms SQL Reference
AWS Clean Rooms API Reference
AWS Clean Rooms ML API Reference
AWS Clean Rooms launches privacy-enhancing synthetic dataset generation for ML model training

Next Month

If you would like to start your new year in a vibrant community, the AWS Community Builders Program will be opening soon in early January 2026 for the annual application intake. You may join the waitlist and be notified by email as soon as the application opens.

Being selected as an AWS Community in your area of expertise enables you to win AWS swag, participate in webinars, join a global community of builders, access AWS certification vouchers and also receive mentorship from AWS experts to level up your knowledge.

10,000 AI Ideas Competition

If you are looking for a challenge and would like to win cash prizes, you may participate now in an AWS AI project over the summer holidays.

10,000 AI Ideas Competition

You may imagine and be creative to build solutions using Kiroand other AI services such as Amazon Bedrock.

Until the next update, happy learning! 😀

AWS re:Invent 2025 recap Part 2 - Data Edition with Frontier Agents

Wendy Wong — Wed, 24 Dec 2025 07:54:06 +0000

Ready to Lift Off

There was something different in the atmosphere for data scientists, AI engineers and Developers at AWS re:Invent 2025 this year with product features aligning with business problems that connect AI innovation with human intelligence. You may also read the Part 1 AWS re:Invent 2025 highlights from my Coffee n Learn newsletter at this link.
If you missed the live stream from Las Vegas you may watch the keynote below, watch innovation talks and breakout sessions from AWS Events on Youtube.

Lesson Outcome

In this lesson you will be able to identify the key announcements from the keynote delivered by VP of Agentic AI, Dr Swami Sivasubramanian and also apply your knowledge to build frontier agents.

Key Announcements

Strands Agents

The open source AI agents SDK on Amazon Bedrock removes complex orchestration and allows you to build an AI agent from your laptop with a few lines of code. Strands has already been downloaded 5 million times.

New features: AWS adds support for both Typescript and edge devices (Preview).

Production: AI agents may be deployed at scale

✓ Removes sensitive data before sharing data with other agents
✓ Includes sophisticated memory context that remembers user profiles
✓ Includes identity and access management to authenticate users and tools to manage credentials
✓ Includes modularity and enables you to observe and debug code

Amazon Bedrock - Episodic functionality in AgentCore Memory

New features from 2 December include AgentCore Policy (Preview) to create guardrails for AI agents and Evaluations (Preview) to give trust for agents in simulated scenarios.

With new feature Episodic functionality in AgentCore Memory launched on 3 December which enables AI agents to remember the past and also provides context for the current situation

Amazon SageMaker AI - AI Model Customization (GA)

New feature: Amazon Sagemaker AI - reinforcement fine-tuning

Additional model customization includes AI modelssuch as Deepseek, Amazon Nova and Qwen for reinforcement fine-tuning to reduce complexity and cost of training a reward model and providing human evaluation.

You may use natural language processing in a prompt to create a use case, the AI model (teacher model) will adjust the base model (student model).

Amazon Nova Forge (GA)

Organizations may build a frontier model with Amazon Nova curated training data, the Nova AI model may be tailored for your industry use case such as healthcare whilst reducing cost.

Amazon Nova Act (GA)

You may build web browser automation for your tasks. The state will be consistent for both training and production.

With reinforcement learning gyms, the AI agents will take action and learn from outcomes.

You may try the Amazon Nova Act playground to experience the UI workflow automation.

What is a frontier agent?

At AWS re:Invent 2025, a new class of AI agents was introduced called frontier agents which are autonomous and can continue working for long periods of time and are scalable to your needs.

AWS introduced three frontier agents:

Kiro autonomous agent(Preview)

Kiro autonomous agent maintains context and assists teams with software development tasks and keeps building knowledge from your interactions.

For example Kiro may help you to update python dependencies from and older version to your latest version and also assist with other software tasks whilst maintaining context aware knowledge so that you do not need to provide new knowledge for each session. Kiro will try to solve the problem on their own without your intervention.

AWS Security agent (Preview)

This is a frontier agent that can provide security of your applications throughout a development lifecycle.
The virtual security agent can perform tasks such as penetration testing and identify security risks.

What are the features?

You may design a security overview
Analyzes your pull requests against your company's security requirements
Provides on-demand penetration testing

What are the benefits?

Validates your security requirements
Provides on-demand access
Scalable security reviews
Provides actionable pull request to remediate security risks
Multi and hybrid cloud support

In this public preview you may access AWS Security agent in the US East (N. Virginia) Region.

3.AWS DevOps agent

This is an AWS frontier agent that helps you to prevent and respond to production incidents.

AWS DevOps agent is already used by Commonwealth Bank of Australia.

What are the features?

The AWS DevOps agent integrates with your existing tools to support your workflows.

You may learn more about configuration from the AWS User Guide.

What are the benefits?

Responds to production incidents quickly
Prevents future incidents
Provides insights into your workloads

Next Month - Start your 2026 with momentum

AWS Community Builders Program 2026 - Application Opens in January 2026

If you are passionate about content creation and being part of a vibrant global AWS Community you may sign up for an AWS Builders ID and register for AWS Builders Center before January 2026 to receive all the latest information for your learning.

You may also place your name on the waitlistto be notified as soon as the application opens.

Join an AWS User Group

If you would like to learn data, AI, cloud, serverless and more - you may join an AWS User Groupin your city. You may join the AWS Builder Center and locate a user group who will support you as you build new skills and participate in AWS Summits and AWS Community days.

Join a Cloud Club

If you are a university student, you may also sign up for an AWS Builders ID and locate a Cloud Club on campus in your city to support you as you learn cloud, data and AI skills.

Until the next update, happy learning! 😀

Wishing everyone happy holidays!

Reimagine an AI-powered contact center - Amazon Q In Connect

Wendy Wong — Sat, 29 Mar 2025 11:05:57 +0000

Preparing to change the landscape of customer service

I've been working in contact centers for about 5 years including my days in finance.

In the US, the advancement of Contact Center as a service is changing how we can anticipate the needs of customers and use AI to transform workforce management and also reporting for supervisors and agents.

Scheduling

Real-time schedulers have problems with agents swapping shifts at the last minute.

Forecasts are rigid and cannot be changed after they are created and published every fortnight.

Supervisors

Supervisors have to maintain SLA and do not have time to provide 1:1 coaching to their team members or have the visibility to be proactive and to change queue priorities.

They do not have the ability to create their own real-time dashboards for day to day operations and to view the status of their agents. They may only view historical records or end of day reporting and cannot take immediate action to anticipate events or an increase in call volumes.

Agents

Agents do not have the flexibility to swap shifts and to manage their burnout.

As they have to maintain CSAT of 98%, adherence and provide customers with a survey after a call, they might not have time to accurately record file notes or wrap-up codes to understand the reason for the customer's call. Wrap-up codes are important for reporting.

Also with knowledge management guides located on intranets and a lack of training for new events, they may lack the information to service their customers.

Amazon Q in Connect

Amazon Q in Connect is an AI powered assistant integrating with Amazon Connect to lighten the load for different contact center personas such as:

Supervisors
Agents
Real-time schedulers

We can have agents working with Amazon Q to help search for information on a call with enterprise knowledge base from documents uploaded to Amazon S3 or perform search with Amazon Kendra.

I have published my new course on LinkedIn Learning to help enterprise adopt AI in their future contact centers to build the future with generative AI to give time back to agents, help them capture and summarize calls and help supervisors produce real-time dashboards of their operations.

In Australia, contact center as a service to integrate data and provide personalized responses to customers is something that one of global contact center providers is anticipating.

However in Amazon Q in Connect you pay as you go and it is easy to create an instance for your virtual contact center to address a few pain points to boost productivity.

To change customer experience in contact centers that is powered by AI and enabled by cloud, we can help organizations have visibility and also a faster response time via search and text summarization.

NEW: LinkedIn Learning course

You are welcome to learn about AI-powered contact center in my new course:

AI-powered contact center with Amazon Q in Connect - Amazon Web Services (AWS) Video Tutorial | LinkedIn Learning, formerly Lynda.com

Join Wendy Wong for an in-depth discussion in this video, AI-powered contact center with Amazon Q in Connect, part of Amazon Q in Connect: GenAI Contact Center.

linkedin.com

Delivering data-driven decisions on AWS

Wendy Wong — Thu, 11 Jan 2024 11:55:45 +0000

Upskilling in 2024

Happy New Year! Over the last six months I have been busy working and teaching.

With my background in consulting, business, data science and analytics I thoughtfully considered the needs of new learners to help them learn AWS in small chunks.

Last week my course was published on LinkedIn Learning Delivering data-driven decisions on AWS: Applying machine learning, data engineering and generative AI

As a teacher at heart, I have personally created this to help data analysts, data scientists, developers, business analysts and data engineers to help them get started on AWS cloud and data.

Deliver data-driven decisions with AWS - Delivering Data-Driven Decisions with AWS: Applying Machine Learning, Data Engineering, and Generative AI Video Tutorial | LinkedIn Learning, formerly Lynda.com

Join Wendy Wong for an in-depth discussion in this video, Deliver data-driven decisions with AWS, part of Delivering Data-Driven Decisions with AWS: Applying Machine Learning, Data Engineering, and Generative AI.

linkedin.com

Ongoing Learning - Data and IT education

If you would like to learn more with me this year you may also subscribe to my weekly newsletter called Coffee N learn. This is a place where I combine technical writing and learning to help you learn more about data analytics, data science and also IT education including AWS cloud.

You may also subscribe here for the Coffee N Learn newsletter

Next Week

Join us next week for AWS re:Invent recap on LinkedIn Audio for Analytics and Databases on Tuesday 16 January at 5pm (AEDT).
You may register to attend LinkedIn Audio at this link

We have an exciting agenda and line up of speakers shown below from AWS Community in APJ.

We will be broadcasting live to allow you to tune in especially if you were not able to attend AWS re:Invent 2023 in Las Vegas or join an in-person event in your city.

You may also watch all the keynotes and sessions on Youtube at this link brought to you by AWS Events channel.

Happy Learning! 😂

Amazon Q in Amazon QuickSight (Preview)- Generative BI with Natural Language Processing

Wendy Wong — Sun, 10 Dec 2023 12:45:11 +0000

What is hot off the press 🪄 from AWS re: Invent 2023?

I am still buzzing from attending the keynotes from Amazon Web Services CEO Adam Selipsky on Day 2 and VP of Analytics, Databases and Machine Learning Dr Swami Sivasubramanianon Day 3 who shared new announcements for Amazon Q.

As a data scientist, data analyst, developer you may be spending up to an hour each day writing production code to integrate and transform multiple datasets, build a data model and then design interactive dashboards to uncover business insights.

Let me introduce you to Amazon Q and we will learn how to incorporate generative AI to create dashboards using natural language processing for supercharging data storytelling.

Lesson Objectives

In this lesson you will learn the following:

What is Amazon Q?
What is Amazon QuickSight?
What are the benefits?
What are the features?
How do I get started?
Build a dashboard using Amazon Q in Amazon QuickSight(Preview)

What is Amazon Q?

Amazon Q is an AI-powered assistant designed for work and can be tailored to your business by taking actions, answering questions, generating content and helping you to solve problems.

Amazon Q in QuickSight is now available in public preview which means that you can design dashboards using natural language processing by using a free-trial period of 30 days and you will not be charged under the two available pricing plans i.e. Amazon Q Business or Amazon Q Builder which is coming soon.

You may find out more from this video below, Introducing Amazon Q

What is Amazon QuickSight?

Amazon QuickSight is a cloud-powered business intelligence service that allows you to build interactive dashboards that includes machine learning insights that provides enterprise security and can easily scale.
You may connect to data from multiple sources including AWS services, spreadsheets and third-party databases.

What are the benefits?

As a data scientist, data analyst or business analyst you will not need to code in any programming language with Amazon Q in Amazon QuickSight.

Some key benefits include:

Reduce time spent to build interactive dashboards to uncover data insights
Ask specific questions using natural language e.g. drill-down or filter data
Amazon Q caters to data and security needs with security in the cloud under the AWS Shared Responsibility Model by understanding your governance requirements for roles and permission levels.
Create visually appealing and compelling narratives

What are the features?

Amazon Q in QuickSight (Preview) includes features that allow you to speed up the delivery of data insights to your business stakeholders by using natural language processing.

The summary of key features include:

Create data stories in minutes by entering prompts in natural language
Obtain key highlights by asking Amazon Q to build an executive summary which takes a few seconds by leveraging large language models.
Produce contextual narratives from questions asked using Amazon Q.
Amazon Q allows you to drill-down into information on the dashboard.
data Q&A Question and Answering capability of Amazon Q allows you to dig deeper into your data by asking questions beyond your dashboard.

How do I get started with Amazon Q in QuickSight?

You may get started with Amazon Q in QuickSight by accessing the free trial in Preview.

If you would like to find out more details, you may watch the session on how to get started with Amazon Q from VP of Amazon Matt Wood at AWS re:Invent 2023:

Tutorial: Build a dashboard using Amazon Q in Amazon QuickSight (Preview)

Let's gets hands on practice to build a dashboard powered by generative AI.

Step 1: Find an interesting dataset that is open-source.

I have selected the Starbucks from Kaggle.com

Step 2: Sign into the AWS Management Console as an IAM user
Step 3: In the search bar type 'Amazon Q'.

Switch to a supported AWS region e.g. US East(N.Virginia)

Step 4: Click Free Trial for Amazon Q in QuickSight

You may select Enterprise + Q which is a 30 day free trial for your team.

Click Continue and enter your contact details.

Step 5: Create your QuickSight Account and under pricing click No. Maybe Later.

Step 6: Select an appropriate Authentication method.

I have an existing AWS IAM credential and I will select the first option.

Secondly, create a QuickSight Account Name and enter an email address.

Step 7: Create a folder in your Amazon S3 bucket and upload the Starbucks csv file.

Step 8: Select the Amazon S3 bucket to access the data.

Click Finish.

Step 9: You will receive a notification message that your Amazon QuickSight account has been created.

Step 10: You will be directed to the Amazon QuickSight welcome page of your account.

Step 11: Click New Analysis.

Step 12: Select New dataset.

Step 13: Connect to data by selecting the option Upload a file and select the Starbucks csv file saved in your local directory.

Step 14: Confirm file upload settings and click Next.

Step 15: Click Edit Settings and Prepare Data to inspect the data and check the correct data types.

Step 16: Click Publish and Visualize.

Click Build Visual and create a topic to ask questions using Amazon QuickSight.

I have named the topic 'Coffee Details' and click Link Topic.

Amazon Q is preparing the topic in QuickSight.

Step 17:The topic 'coffee details' has successfully been created and we will navigate to the Q bar.

Step 18: We will start by asking a few questions in Q bar.

Amazon Q has recommended a few suggested questions to ask before I start typing the business question.

Step 19: I click on 'Coffee Details', there is no narrative provided as this is not a question.

Step 20: I click Ask Q and select a suggested question:

'total cholesterol mg by beverage?'

Step 21: I click Ask Q and select a suggested question:

'Top 5 beverages by total calories'

Step 22: I click Ask Q and select a suggested question:

'Beverage category with the highest total fat by category'

Step 23: Click Ask Q to select a suggested question:

'Beverage category with the highest total sugar by category'

Step 24: Navigate to Named Entity and group variables.

I have grouped trans fat, saturated fat and total fat.

Step 25: I have created a second named entity for 'Vitamins'.

Step 26: I have created a third named entity for 'Sugar' by entering at least two dimensions and clicking Save.

Step 27: I have created a fourth named entity for 'Nutrition value' by entering at least two dimensions and clicking Save.

Step 28: We can navigate back to Ask Q such as the total amount of sugar in each beverage and click the light bulb icon to view insights.

Step 28: Five visuals have been saved as pinboard.

To build a dashboard, you may select Add to analysis and choose four pinboard visuals created by Amazon Q.

You may then check the boxes for Allow executive summary and publish the dashboard.

Step 29: Let's create a data story.

Navigate to Data stories on the left-handside and select New story. Select Slideshow.

Enter a 'Story title' and describe the story in simple natural language by entering a prompt and Add Visuals.

Amazon Q was able to generate content via Data stories to create a slide presentation deck based on my visuals with AI-powered data insights.

Step 30: By navigating to my dashboard, on the right-handside under Build select from the drop-down menu Executive Summary and click on Sheet 1.

In a few seconds, content is generated for an Executive Summary as shown below:

Conclusion

You may easily create a dashboard using generative AI to supercharge data storytelling with large language models using Amazon Bedrock to generate insights, suggested questions and even help you create a slide deck with data stories.

An executive summary is content created from your visuals and natural language prompt that you provide on a selected worksheet.
It is very easy to get started under Preview with the 30 day free-trial.

Until the next lesson happy learning! 😁

Resources and Amazon QuickSight Community

You may watch this video to learn how to use Generative BI with Amazon Q to build dashboards in Amazon QuickSight:

If you would like to learn more, be sure to have a look at Amazon QuickSight Community:

Amazon Q November 28 -- Announcements at AWS re:Invent 2023 - What's New - Amazon Quick Community

Amazon Q Announcements, November 28 at AWS re:Invent 2023 New Amazon Q in Quick Sight uses generative AI assistance for quicker, easier data insights (Preview) Now you can experience the Generative BI capabilities in Amazon Quick Sight announced on July 26, as well as two additional capabilities for business users. Amazon Q in Quick Sight simplifies data exploration with Generative BI capabilities (Preview) Amazon Quick Sight announces three new natural language cap...

community.amazonquicksight.com

Did you miss AWS re: Invent 2023?

You may watch these highlight sessions discussing Amazon Q from AWS re:Invent 2023 last week from Las Vegas. I recommend the following to watch:

Senior Principal Engineer, Clare Liguori - Amazon Q: Your new assistant and expert guide for building on AWS

CEO at Amazon Web Services, Adam Selipsky's Day 2 keynote, watch Amazon Q announcements from timing [01:22]

References

Next Week - Join an AWS re:Invent 2023 recap in Australia or New Zealand.

You may find more details for Re-cap sessions happening this week in your local city.

If you are located in Sydney, you may register to attend AWS re:Invent 2023 recap on Thursday 14 December this link

500 million+ members | Manage your professional identity. Build and engage with your professional network. Access knowledge, insights and opportunities.

linkedin.com

Get started with Amazon Bedrock for Image Generation - Part 1 Generative AI

Wendy Wong — Sun, 08 Oct 2023 12:47:11 +0000

Amazon Bedrock is now GA

Amazon Bedrock is a fully managed service that helps you to build generative AI applications with foundation models from AI leaders such as AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon.

Amazon Bedrock is now generally available since 28 September 2023. In the latest announcement, you will now be able to access Llama 2 and Meta's large language models via Amazon Bedrock API and also use the AWS CLI.

Amazon Bedrock announced by AWS in public preview in April 2023 and also announced at AWS re:Inforce 2023 with the talk Securely build generative AI apps & control data with Amazon Bedrock (APS208).

Learning Objectives

Identify the features of Amazon Bedrock
Explore the benefits of Amazon Bedrock
Identify the common use cases of Amazon Bedrock
Understand the solution architecture of Amazon Bedrock
Understand pricing and AWS region support
Identify real world industry applications of Amazon Bedrock
Generate an image using AWS Management Console

Why Amazon Bedrock?

Amazon Bedrock is serverless which means that you do not need to manage any IT infrastructure.
You may experiment with foundation models without writing any code.
You may use your own data and customize foundation models with fine-tuning or retrieval augmented generation (RAG) and use agents to perform complex business tasks.

What are the benefits of Amazon Bedrock?

You may access foundation models from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon to build generative AI applications using playgrounds for experiments and also use Amazon Bedrock API to make inference.
You may customize the foundation model using your own dataset that can be uploaded into Amazon S3 for both training and testing.
You may build agents to perform complex business tasks
HIPAA eligibility and GDPR compliance
Amazon Bedrock data security ensures that your data in transit and at rest is encrypted allows the use of access keys.

Amazon state that:

You can use AWS PrivateLink with Amazon Bedrock to establish private connectivity between FMs and your Amazon Virtual Private Cloud (Amazon VPC) without exposing your traffic to the Internet.

What are the features of Amazon Bedrock?

This is an overview of Amazon Bedrock below.

The features are outlined in the Amazon Bedrock User Guide and include:

Text playground – A hands-on text generation application in the AWS Management Console.

Image playground – A hands-on image generation application in the console.

Chat playground – A hands-on conversation generation application using the console.

Embeddings – Use the API to generate embeddings from the Titan Embeddings G1 - Text model.

Examples library

You may explore example cases from the examples library.

What are the AWS supported regions?

The supported regions to get started include US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore) and Asia Pacific (Tokyo).

What is the Pricing for Amazon Bedrock?

There are two pricing models for Amazon Bedrock which are:

On Demand pricing: You pay for your usage and there is no time commitment.

You may refer to details for on-demand pricing here:

Text generation models: you are charged for every input token processed and every output token generated.
Embeddings models: you are charged for every input token processed. A token is comprised of a few characters and refers to the basic unit that a model learns to understand user input and prompt to generate results.
Image generation: you pay for the number of images generated.

Provisioned Throughput pricing: You pay for the time commitment with purchased throughput to run inference on models.

Amazon recommend that provisioned throughput is suited to large workloads.

With Amazon Bedrock pricing, you pay to run inference on any of the third-party foundation models. Pricing is based on the volume of input tokens and output tokens, and on whether you have purchased provisioned throughput for the model.

Provisioned Throughput is charged by the hour, you have the flexibility to choose between 1-month or 6-month commitment terms.

Foundation Models

Base Model

There are a variety of base foundation models for you to choose from that are subject to change. At the moment, you may access the following models:

Amazon Bedrock supports the following models:

AI21 Labs: Jurassic-2 Ultra and Jurassic-2 Mid
Amazon: Titan Text G1 - Express (limited preview) and Titan Embeddings G1 - Text
Anthropic: Claude v1.x, Claude v2.x and Claude Instant v1.x
Cohere: Command
Stability.ai: Stable Diffusion XL 0.x and Stable Diffusion XL 1.x are in limited preview and you may contact your AWS account manager for more information.

Custom Model

You may also bring in your own dataset to customize the models
with hyperparameters epochs, batch size, learning rate, warmup steps to fine tune the model.

You may build a foundation model with training data and deploy a fine-tuned model with Amazon Bedrock API.

What are the common use cases?

Text generation e.g. write essays and blog posts
Virtual Assistants e.g. take user request and perform the tasks
Chatbot: for question and answering
Search e.g. Search for information from within a document
Text Summarization e.g. Summarize the main theme of a book or document
Image Generation e.g. generate real-life photos for property brochures

What are the real-world industry applications?

There are a few customer stories from AWS:

Adidas: Used Amazon Bedrock to create a generative AI application for helping their community search for answers to questions.
Booking.com: Used Amazon Bedrock to build a generative AI application to send destination and accommodation recommendations that was personalized to their customers.
Salesforce.com: Salesforce have partnered with AWS to bring your own large language model using Salesforce Data Cloud, this was announced at Salesforce Dreamforce on 11 September 2023.
Genesys: Genesys are adopting generative AI, to allow users to access a variety of large language models, such as Genesys-developed models and multiple third-party foundational models through Amazon Bedrock.

Solution Architecture

The solution architecture for image generation using Stable Diffusion on Amazon Bedrock is from the source Amazon Bedrock workshop available on Github

Tutorial: Getting started with Amazon Bedrock with Image Playground

Step 1: Navigate to the AWS Management Console. Login to AWS account as an IAM Admin User.

Step 2: In this tutorial we will use AWS region N Virginia (US-east-1).

Step 3: Type the word 'Bedrock' into the search bar and navigate to the Amazon Bedrock console and click Get started.

Step 4: Request model access. Navigate to the third party provider of foundation models e.g. Stability AI and select edit access.
Check the box to select the foundation model and Request Access.
It will take a few minutes to have access granted. Refresh your browser.

I selected the foundation model from Stability AI which was Stable Diffusion XL-Preview.

Step 5: Navigate to Image to select the image playground to commence prompting.

Select the image on the left-hand pane.

Type a word in natural language which is to give a prompt. In the box you may type a few words e.g. Sydney Opera House Real Life Summer.
It will takes a few seconds to create an image.

On the right-handside of the image playground, you may adjust the slider to update the inference configuration to change the output quality of the generated image.

Step 6: You may also download this image that you have generated and use this for your project e.g. blog post, newsletter, post card, document, magazine etc.

Step 7: Clean Up Resources

As a best practice, if you no longer need to generate images from your foundation model, please clean up your resources by removing model access from the third party foundation model provider by clicking edit, un-check the box e.g Stability AI and then select Save Changes.

Resources

If you would like to learn more about Amazon Bedrock, I encourage
you to explore to useful resources.

AWS Skill Builder

You may subscribe for a 7-day free trial of AWS Skill Builder for a limited time and complete the 1-hour course Amazon Bedrock Foundations course.

Amazon Bedrock Workshop - Github

You may also obtain practical hands-on knowledge of Amazon Bedrock via workshops on Github you will be able to practice with the labs.

Conclusion

In this lesson you have learnt how to generate images with foundation models. I hope you will continue to apply prompt engineering to your own use cases.

Until the next lesson - Happy Learning! 😀

References

This Month - AWS Innovate Modern Application - 26 October 2023

You are invited to AWS Innovate Modern Application on 26 October 2023 in the Asia Pacific and Japan regions.

This conference is FREE, register for the keynotes and sign up for various sessions to learn the latest in Generative AI and more.

You may register at this link.

Next Month - AWS re:Invent 2023 - 27th November to 1st December 2023

In-Person experience

You are invited to AWS re:Invent 2023 in Las Vegas, you may register at this link for an in-person experience. After you register you may navigate to the session catalog and start selecting your sessions and favourite the breakout sessions, chalk talks, keynotes, innovation talks, customer talks, community events and more.

AWS re:invent is also about building your networks and connecting with the AWS Community at PeerTalks

If you need more help with selecting sessions from the catalog, be sure to browse the attendee guides that were written by AWS Heroes and written for you in mind to help you find recommended sessions from generative AI, machine learning, data and analytics, containers, community, devtools, serverless, reliability engineering, storage, sustainability, introverts and enterprise leadership.

You may also browse the agenda to help plan your week.

Live Stream Experience

If you can't make it to Las Vegas this year, you may register for FREE and attend the live stream so that you may also enjoy the keynotes, innovation talks and your free pass will provide you will post conference access to selected breakout sessions. You may register for live streaming.

Amazon Bedrock announcement - 29 September 2023 by Dr Swami Sivasubramanian the VP of Analytics, Database and Machine Learning

// Detect dark theme var iframe = document.getElementById('tweet-1707412406102634826-842'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1707412406102634826&theme=dark" }

Getting started - Amazon Redshift Serverless automatic mounting of AWS Glue Data Catalog

Wendy Wong — Sun, 30 Jul 2023 10:58:50 +0000

Run queries from your data lakes

Amazon Redshift announced a new feature on 25 July 2023 that allows developers, data engineers, data analysts and data scientists to run SQL queries easily using Query Editor v2 in a provisioned environment or Amazon Redshift Serverless by querying data from AWS Glue Data Catalog.

Lesson Objectives

In this lesson, you will learn how to:

Change the system-level configuration of the data catalog auto mount.
Show a list of schemas in the database named in the awsdatacatalog.
Show a list of tables in a schema.
Show a list of columns in a table.

How does this work?

This new AWS service integration means that developers and analysts will no longer have to create an external schema from Amazon Redshift in order to be able to query data in data lakes (i.e. Amazon S3 bucket) from AWS Glue Data Catalog.

I have included an image below to demonstrate that an external schema will no longer need to be created.

Hence, you will be able use Query Editor v2 to query data that is cataloged in your AWS Glue Data Catalog.

(Note: Querying the AWS Glue Data Catalog is not available in all Amazon Redshift Regions).

Solution Overview

This is my interpretation of the solution architecture with this new integration between Amazon Redshift Serverless to query directly from AWS Glue Data Catalog using Query Editor v2.0.

Pre-requisites

If you would like to learn more about getting started using AWS Glue and AWS Glue Studio you may read my previous blog posts below:

Tutorial: Querying the AWS Glue Data Catalog using Amazon Redshift Serverless.

Step 1: Sign into your AWS account.

If you do not have an AWS account you may create one here.

Step 2: Sign into the AWS Management Console as an IAM Admin User.

If you do not have an IAM Admin user role you may create one here.

Step 3: Navigate to the AWS Glue dashboard.

I have previously created a database called 'community'.

I have a table called 'aws_community_builders_apj' which I cataloged in the AWS Glue Data Catalog last year.

Step 3: I already created an instance of Amazon Redshift Serverless by navigating to the Amazon Redshift dashboard.

If you wanted to know how to get started with Amazon Redshift Serverless you may read my previous blog here.

Step 4: Once an instance of Amazon Redshift Serverless is created, click on the namespace. In this example, I will click 'default-namespace'.

Step 5: Navigate to the left-handside menu and select 'Query Editor v2'.

Step 6: Double click on serverless-workspace and enter your database credentials as a Federated User. Enter the database name, username and your password.

Step 7: In Query Editor v2 enter these SQL commands.

This command will display whether awsdatacatalog is mounted for the Amazon Redshift Serverless data warehouse.

SHOW data_catalog_auto_mount;

Step 8: ALTER command will change the system level configuration when Amazon Redshift Serverless is paused or resumed.

ALTER SYSTEM SET data_catalog_auto_mount = on;

Step 9: Show a list of schemas that is cataloged in the AWS Glue Data Catalog.

SHOW SCHEMAS FROM DATABASE awsdatacatalog;

Step 10: Show a list of tables in the schema for community.

SHOW TABLES FROM SCHEMA awsdatacatalog.community;

Step 11: Show a list of columns within the table.

SHOW COLUMNS FROM TABLE awsdatacatalog.community.aws_community_builders_apj;

Step 12: If you would like to query the AWS Glue Data Catalog, you will have to grant your user the necessary permission. e.g.The IAM user role to access the database.

GRANT USAGE ON DATABASE awscatalog to'IAM:myIAM USER"

Conclusion

As organizations innovate and wish to analyze all of their data to provide quick answers to their business questions, we have learnt how to empower developers, data analysts, data scientists and business analysts to access data from the AWS Glue Data Catalog without creating an external schema and using Query Editor v2 from Amazon Redshift Serverless or a provisioned data warehouse to generate SQL queries with insights in seconds.

Resource

You may read more about this new capability in the AWS Big Data Blog.

Reference

Until the next lesson, happy learning! 😀

Last Week - AWS Summit New York, 26 July 2023 keynote VP of analytics, databases and machine learning, Dr Swami Sivasubramanian

You may watch the AWS Summit New York keynote from VP of analytics, databases and machine learning Dr Swami Sivasubramanian.

You may also read all the latest announcements and highlights from analytics, database and Generative AI here.

Last Week - AWSonAir on Twitch, 26 July 2023

Watch on-demand on AWSonAir on Twitch below, the latest announcements including AWS Glue from AWS Summit New York.

// Detect dark theme var iframe = document.getElementById('tweet-1684185369846358017-161'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1684185369846358017&theme=dark" }

Next Month - AWS Innovate Data Edition - 30 August 2023

You may join us and register for AWS Innovate, Data Edition in APJ on 30 August 2023. You may register here and how to unlock the value of data and develop your data strategy as you hear from AWS Experts.

How to check for quality? Evaluate data with AWS Glue Data Quality

Wendy Wong — Fri, 07 Jul 2023 09:47:56 +0000

Data is the new oil

The Women in Data Science (WiDS) Conference 2017 trailer from Stanford University aimed to inspire the audience in the field of data science.

In the trailer, the panelist mentioned the power of data.

Who gets to control data, controls the society that's the reality. -- Bell Wei (Carolyn Guilding Chair in Engineering and Innovative Learning, San Jose State University)

Having access to the right data and understanding your data will empower you to make data-driven decisions for your organization. Data understanding is Step 2 of the Data Analytics and Data Science workflow called Cross-industry standard process for data mining (i.e. CRISP-DM).

You may read more about CRISP-DM here. The second step:

Data understanding – What data do we need or is available? Is it clean?

This is the data science workflow using CRISP-DM explaining all the stages and the image is from Wikipedia:

Learning Objectives

In this lesson you will learn the following:

How to create rules with DQDL
What is AWS Glue?
What is AWS Glue Data Quality?
Solution Architecture
How to check for data quality results?

Who owns data quality?

Is data quality the responsibility of the data engineer building your ETL data pipelines? Is it the responsibility of the data analyst building the data modelling? Or the data scientist building hypotheses and machine learning models responsible for data quality?

Take a look at this Twitter post on data quality here.

In fact we can all play part to ensure that with good quality data, we can improve our data analysis and machine learning model performance to make business decisions.

What is AWS Glue?

AWS Glue is an AWS service that allows data engineers, business analysts, data analysts, developers and data scientists to integrate data from multiple sources and also perform ETL.

It is serverless data integration service to allow you to easily scale your workloads in preparing data and moving transformed data into a target location.

You do not need to provision any servers or clusters before using AWS Glue with a managed serverless experience.

You may bring in your own code or notebook to create an ETL job on demand, batch or streaming. You may create an AWS Glue data catalog to make data available for others to use.

You may also use AWS Glue Studio a visual editor to create your ETL pipeline.

What is AWS Glue Data Quality?

AWS Glue Data Quality is a new product feature of AWS Glue that is now generally available since 6 June 2023.

AWS Glue Data Quality allows you to measure and monitor 'good or bad data' in your ETL pipelines before it enters your data lake or data warehouse to ensure high quality data is available to make data-driven decisions.

What are the benefits?

In the AWS Glue Developer Guide for AWS Glue Data Quality, the benefits include the following:

Serverless – there is no installation, patching or maintenance.
Get started quickly – AWS Glue Data Quality quickly analyzes your data and creates data quality rules for you.
Improvise your rules to check the integrity and accuracy of the data
Evaluate data quality and make confident business decisions
Zero in on bad data with errors
Pay as you go - There are no fixed costs and you pay for your usage when you use AWS Glue Data Quality
Data quality checks - you can implement checks in the AWS Glue data catalogue
You may create rules to check the profile of your dataset.
You may get started quickly

Solution Architecture

Below is my own solution overview of the new product feature of AWS Glue Data Quality.

Dataset

Let's examine the Amazon Data Science Books Dataset from Kaggle.com

Pre-requisite

You may read this blog to learn how to get started with AWS Glue Data Quality.
You may read this blog How to ETL with AWS Glue - Part 1.
You may dive deeper with a practical example in Part 2.
You have an AWS Account, if you do not have an account you may learn how to create one here.

Tutorial 1: Add the Evaluate Data Quality transform to the visual job in AWS Glue Data Studio

In this tutorial you may refer to the instructions from the AWS Glue User Guide.

Step 1: Log into your AWS account as an IAM Admin User.

Step 2: Navigate to Amazon S3 and create your bucket in an AWS region of your preference and click Create bucket.

Step 3: Upload the Amazon data science books dataset into your Amazon S3 bucket and click Upload.

Step 4: Navigate to AWS Glue dashboard.

Click Create a job to evaluate data quality

Step 5: Click on Visual and select Evaluate Data Quality.

Step 6: Add the data quality node.

On the AWS Glue Studio console, choose Visual with a source and target from the Create job section. Choose Create.

Step 7: Choose a node on which to apply the data quality transformation. Currently there is no node selected.

You may select the transform node (i.e. Transform - ApplyMapping).

Step 8: On the left-hand side click the blue plus sign and select from the drop-down menu Evaluate Data Quality towards the bottom section. You may name this job and be sure to save it e.g. GlueDataQuality_Tutorial.

Step 9: The selected Evaluate Data Quality transform node will be displayed in the visual editor.

On the right-hand side you may inspect if you would like to retain the current parent node or change it from the drop-down menu. (Note: The parent node is connected to the Evaluate Data Quality node).

Step 10: You may validate data quality rules across multiple datasets. The rules that support multiple datasets include:

Referential Integrity
DatasetMatch
SchemaMatch
RowCountMatch
AggregateMatch

When you add multiple inputs to the Evaluate Data Quality Transform, you need to select your primary input.

Select your primary input which is the Amazon data science books dataset to validate data quality for.

All other nodes or inputs are considered as references.

Use Evaluate Data Quality transform to identify specific records that failed data quality checks.

(Note: New columns flagged as bad records are added to the primary dataset).

Step 11: In the Data Source - S3 bucket select the S3 bucket where the dataset or primary input is saved.

Step 12: Click on Output schema tab to modify any data types e.g. change price from string to integer.

Click Apply to change the data types.

Step 13: Select the Data target - S3 bucket and save the transformed dataset in this location.

Tutorial 2: Create rules using DQDL builder

Step 1: Preview the Amazon data science books dataset and let's create a rule using the DQDL rule builder to check for the completeness of data.

You may browse the available data quality rules from the Rules type tab which include the following:

ColumnCount
ColumnLength
ColumnExists
ColumnDataType
ColumnValues
ColumnNameMatchesPattern
Completeness
CustomSql
DataFreshness
DatasetMatch
DistinctValuesCount
Entrophy
IsComplete
IsPrimaryKey
Sum
Uniqueness
ReferentialIntegrity
Mean
RowCount
RowCountMatch
StandardDeviation
UniqueValueRatio
SchemaMatch

I selected the data quality rule for Completeness, because I would like to check the percentage of missing data greater than 80% for the following variables:

Price
Price (that includes used books)
Number of book reviews
Average rating

Firstly preview the primary data source to understand the data.

The following rules were created in the DQDL Builder in the * Schema* tab.

Rules= [
    Completeness "avg_reviews"> 0.8, Completeness "n_reviews"> 0.8, Completeness "price" > 0.8, Completeness "price (including used books)" > 0.8
]

The Completeness rule will check for the specified columns if there is greater than 80% of non-null values present in the primary data source.

Tutorial 3: Configure Data Quality Outputs

Step 1: After the data quality rules are created, you can select additional options to be included in the data quality results output.

I have selected two additional options:

Action: publish actions to Cloudwatch
Data Quality Results: to flag fail or pass results

Below is an image of the rule outcomes node.

Step 2: Under Data Quality Transformation Output, I also checked the box for Original data as this will also append additional columns to the primary dataset to indicate bad errors.

Tutorial 4: Configure Data Quality actions

After a data quality rule is created, you may select actions for CloudWatch to publish metrics or stop jobs based on a criteria.

Actions in CloudWatch are also published to Amazon Eventbridge and can be used to create alert notifications.

On ruleset failure
Fail job after loading data to target
Fail job without loading to target data

Tutorial 5: View data quality results

Click Save and initiate the AWS Glue job by selecting Run.

You may click Run details to inspect the progress of the Glue job.

After the job has completed, select the Data Quality New tab to inspect the results.

You will be able to see that the data quality rules have passed successfully and you may click Download results as a csv file

[
    {
        "ResultId": "dqresult-xxxxxxxxxxxxxxxxxxxxxxxxxx",
        "Score": 1,
        "RulesetName": "EvaluateDataQuality_nodexxxxxxxxxx",
        "EvaluationContext": "EvaluateDataQuality_nodexxxxxxxxx",
        "StartedOn": "2023-07-07T08:57:48.117Z",
        "CompletedOn": "2023-07-07T08:58:08.203Z",
        "JobName": "GlueDataQuality_tutorial",
        "JobRunId": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
        "RuleResults": [
            {
                "Name": "Rule_1",
                "Description": "Completeness \"avg_reviews\" > 0.8",
                "EvaluatedMetrics": {
                    "Column.avg_reviews.Completeness": 1
                },
                "Result": "PASS"
            },
            {
                "Name": "Rule_2",
                "Description": "Completeness \"n_reviews\" > 0.8",
                "EvaluatedMetrics": {
                    "Column.n_reviews.Completeness": 1
                },
                "Result": "PASS"
            },
            {
                "Name": "Rule_3",
                "Description": "Completeness \"price\" > 0.8",
                "EvaluatedMetrics": {
                    "Column.price.Completeness": 1
                },
                "Result": "PASS"
            },
            {
                "Name": "Rule_4",
                "Description": "Completeness \"price (including used books)\" > 0.8",
                "EvaluatedMetrics": {
                    "Column.price (including used books).Completeness": 1
                },
                "Result": "PASS"
            }
        ]
    }
]

If you are running multiple data quality jobs, you may filter the data quality results by date and time.

If you navigate to the tab Script you will be able to see that AWS Glue Studio automatically created Python code for the transformation steps that you could easily download for reusability.

Conclusion

You have learnt how to set up data quality rules in AWS Glue Data Quality using the visual editor of AWS Glue Studio. You have explored how to create an ETL job and examine the data quality results for a pass or fail from rules that were created.

Until the next lesson happy learning! 😀

AWS Glue Data Quality Quick Start videos on Youtube

If you would like to learn more you may watch the following videos

AWS Glue Data Quality Overview

Measure and Monitor Data Quality of your Datasets in AWS Glue Data Catalog

Introducing AWS Glue Data Quality for ETL Pipelines

Reference

New announcements from AWS Glue

Next Week: AWS Builders Online Series - 13 July 2023

You may register to join AWS Builders Online Series and learn from AWS experts on architectural best practices.

Amazon CodeWhisperer with JupyterLab extension for Amazon SageMaker Studio - Part 4

Wendy Wong — Sun, 25 Jun 2023 14:24:40 +0000

What's new with Amazon CodeWhisperer ?

On 10 May 2023, Senior Principal Technologist at AWS Brian Granger announced extensions to generative AI on Jupyter including:

Jupyter AI, an open-source project to bring generative AI to Jupyter notebooks
Introducing new Jupyter extensions to build, train, and deploy ML at scale
Amazon CodeWhisperer Jupyter extension

For developers, cloud architects, data scientists, data analysts and machine learning engineers in particular this is exciting news allowing JupyterLab users to install Amazon CodeWhisperer to use Python notebooks and also Amazon SageMaker Studio.

You may read all of the Jupyter extensions here.

Learning Objective

You will learn:

How to install Amazon CodeWhisperer with JupyterLab extension to generate code recommendations within Amazon SageMaker Studio

What's the benefit of using Amazon CodeWhisperer?

You may get started and up and running on a machine learning project and increase your developer productivity with auto-completion, provides code suggestions by learning from code snippets and python comments.
You may also code in other languages such as:

Python, Java, JavaScript, TypeScript, C#, Go, Rust, PHP, Ruby, Kotlin, C, C++, Shell scripting, SQL, and Scala in various IDEs.

There are built-in security scans as you code based on the AWS Shared Responsibility Model.
Amazon CodeWhisperer is free for individual use.

You may read my previous blog post if you would like to know how to install Amazon CodeWhisperer for VS Code IDE.

I recently used Amazon CodeWhisperer to kickstart a personal machine learning project which allowed me to code faster.

By using Amazon CodeWhisperer in VS code editor, I was able to complete exploratory data analysis (EDA) in python, I simply typed words into VS code editor and asked a question like a friend using natural language. EDA was completed within 15 minutes and learned about my intent by reading my python comments and making the next suggestion in the data analytics workflow.

Get code recommendations for Amazon SageMaker Studio

Amazon SageMaker Studio launched in 2019 as an IDE using JupyterLab
to allow data science teams to collaborate on the end to end data science workflow which includes build models, train models, fine-tune hyper parameters, debug, deploy, and monitor model drift.

On the 10 May 2023, there were new announcements

You may now receive real-time code recommendations from Amazon CodeWhisperer as you code and build your machine learning pipeline within Amazon SageMaker Studio.

Pre-requisites

Creating your first IAM admin user and User group in your AWS account
You may read the following blog to onboard Amazon SageMaker Domain.
Add the Amazon CodeWhisperer-related permissions to your SageMaker execution role (e.g. Amazon SageMaker FullAccessrole)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CodeWhispererPermissions",
      "Effect": "Allow",
      "Action": ["codewhisperer:GenerateRecommendations"],
      "Resource": "*"
    }
  ]
}

Tutorial: Add the CodeWhisperer-related permissions to your SageMaker execution role.

For this tutorial please follow the steps from the Amazon CodeWhisperer Developer Guide.

Step 1: Login to your AWS account as an Admin User and choose your preferred AWS region e.g. Asia Pacific Sydney (ap-southeast-2).

Step 2: Navigate to the IAM dashboard.

Step 3: Create an IAM policy that contains the following actions to enable Amazon CodeWhisperer for Amazon SageMaker Studio role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CodeWhispererPermissions",
      "Effect": "Allow",
      "Action": ["codewhisperer:GenerateRecommendations"],
      "Resource": "*"
    }
  ]
}

Navigate to Policies on the left-handside menu.

Step 4: Click Create policy.

Step 5: Click json.

Step 6: Copy the new IAM policy into the json statement and click Next.

Step 7: Provide a policy name e.g. Attach_Amazon_CodeWhisperer_to SageMaker_Studio and click Create policy.

The policy was successfully created.

You may inspect the policy.

Navigate to IAM and click Roles, search for 'Sagemakerfullaccess role' and check if there are any policies already attached.

Step 8: Navigate to Policies and search for policy 'Attach_Amazon_CodeWhisperer_to SageMaker_Studio' and attach the following IAM entity:

SageMakerfullaccess

Step 9: Navigate to the Amazon SageMaker dashboard and on the left-handside menu click Getting Started and on the right select Create a role.

Step 10: Enter the following details such as providing a role name with the pre-fix of SageMaker. e.g. DataScientist, MLEngineer

DataScientist_SageMaker

You may also provide a short description of the role e.g. create a role to enable Amazon CodeWhisperer recommendations for Amazon SageMaker Studio.

Retain the other default settings and click Next.

Step 11: Configure the role by selecting the ML activities.

You may select more activities or de-select the ML activities that you do not require.

Step 12: Scroll down and enter the name of one of your Amazon S3 buckets to store your training and test data. Click Next.

sagemaker--<AWS account number

Step 13: In the search bar, start typing the name of the IAM policy that was created in Step 7.

Attach_Amazon_CodeWhisperer_to SageMaker_Studio

Check the box of the policy and click Next.

Step 14: Inspect and review the role details.

Review the additional policy attached to the SageMaker execution role and click Submit.

Step 15: The Sagemaker_DataScientist_SageMaker role was successfully created and you may view the role in IAM.

You can see that this new role has an additional policy attached to enable Amazon CodeWhisperer to be used under the SageMaker execution role for Amazon SageMaker Studio.

Step 16: Enable Amazon CodeWhisperer within Amazon SageMaker Studio

Select Studio on the left-handside menu and click Open Studio.

Step 17: In Amazon SageMaker Studio, click Open Launcher.

Step 18: Under the heading 'Utilities and file', click System terminal.

Step 19: Copy the code into terminal:

conda activate studio
pip install amazon-codewhisperer-jupyterlab-ext
jupyter server extension enable amazon_codewhisperer_jupyterlab_ext
conda deactivate
restart-jupyter-server

The installation process will take a few seconds to complete.

Step 20: Navigate to the Launcher homepage and select Create Notebook.

I received the following notification message.

Step 21: Close the Amazon SageMaker Studio session.

Refresh your browser and navigate to Studio and click Open Studio and wait for Amazon SageMaker Studio to open.

Under the notebook you will be able to see a tick against CodeWhisperer to confirm that code recommendations are enabled for the code cells.

Step 22: Launch Studio again and you will be taken to JupyterLab which will open the last notebook and start typing Python code into the code cells to generate code recommendations.

Reference

Until the next lesson, happy learning! 😀

Build on Generative AI - Using your Python scripts in Sagemaker

You may be interested to learn more from to change your Python scripts to run on SageMaker Training? 🦅Join guest Giuseppe Angelo Porcelli, along with cohosts Darko Mesaroš & Emily Webber to learn more on Build on Generative AI

Last week - AWS re:Inforce 2023

You may be interested to learn more about Amazon Bedrock, foundation models and Generative AI from AWS re:Inforce announcements last week. You may watch the keynote from AWS CISO CJ Moses, break-out sessions and leadership sessions here

Image Classification for Natural Disaster Damage with Amazon SageMaker Jumpstart

Wendy Wong — Mon, 19 Jun 2023 16:48:17 +0000

Recovery in the aftermath of a natural disaster

Northern New South Wales experienced multiple floods in 2022, the town of Lismore did not have measures to predict or prevent natural disasters in the low-lying flood-prone region.

After the 2022 floods, many communities were displaced and homes were declared uninhabitable.

There are a few questions to consider in the aftermath:

How do government to help citizens find temporary accommodation? How do they assess photos of flood damage?
How can insurance companies assess photos of flood damage to homes and businesses for residential and commercial claims?

Machine learning can be used to explore patterns instead of relying on human intuition to classify images.

Learning Objectives

In this lesson you will learn how to:

Onboard Amazon SageMaker Domain
Set up Amazon SageMaker Studio
Import a Jupyter Notebook
Import your own dataset or use an existing dataset
Train and deploy an image classification model using Amazon SageMaker Jumpstart
Clean up resources

What is Amazon SageMaker Jumpstart?

You may train and deploy a machine learning model to solve your business problems with pre-trained and open-source models with a few clicks. You may also use SageMaker Python SDK for programmatic access.
SageMaker Jumpstart Industry notebooks can only be run on Amazon SageMaker Studio.

What are the benefits?

You may build, experiment and deploy machine models with:

Foundation models

You may access pre-trained foundation models for your business use case. This allows you to browse and choose from a large library of model providers, build, experiment, customize and deploy your Generative AI models.

Built-in algorithms and pre-trained models

You may access built-in algorithms and pre-trained models to help you achieve specific tasks such as vision, text, audio and documents. The models can be fine-tuned and also deployed quickly especially if you need to test and build from an idea or productionise your machine learning models.

Solution Templates

You may access pre-built ML solutions, browse the solution templates, select a template that includes for your use case and customize by using your own data and deployment is a click of a button.

Share machine learning artifacts

What are common use cases for Amazon SageMaker Jumpstart?

Amazon SageMaker Jumpstart suggests the following use cases:

Demand forecasting
Credit rating prediction
Fraud detection
Computer vision
Extract and analyze data from documents
Predictive maintenance
Churn prediction
Personalized recommendations
Reinforcement learning
Healthcare and life sciences
Financial pricing
Causal inference

Solution Architecture

This is my proposed diagram of using Amazon SageMaker Jumpstart with pre-trained ML solution templates that are initiated with Amazon CloudFormation.

Dataset

The Hurricane dataset was provided in the Financial Services Lab for damage classification.

Pre-requisites

You will need to have an existing AWS account or you may create one here.
You will need to login as an Administrative User and Group

Ensure that your IAM permissions for Administrative User allows you access to the following AWS services:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sagemaker:*"
            ],
            "Resource": [
                "arn:aws:sagemaker:*:*:domain/*",
                "arn:aws:sagemaker:*:*:user-profile/*",
                "arn:aws:sagemaker:*:*:app/*",
                "arn:aws:sagemaker:*:*:flow-definition/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "servicecatalog:*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Onboard Amazon SageMaker Domain

Tutorial 1: Onboard to Amazon SageMaker Domain

Before you start using Amazon SageMaker you will also have to onboard SageMaker Domain.

Step 1: Navigate to Amazon SageMaker and select the AWS region that you prefer to use. I will select AWS region Asia Pacific ap-southeast-2.

Step 2: Choose Domains on the left-hand side menu.

Step 3: Select Create domain.

Step 4: Select Quick Setup on the left-hand side menu.

Step 5: Create a unique domain name.

Step 6: You may use the default name under User profile or you may create a unique name.

You may select from the drop-down menu, the execution role SagemakerFullAccessrole and click Submit.

If you do not have this execution role, you may create one here.

Also, ensure that the box is checked for enable 'SageMaker Canvas permissions' and finally click Submit.

Step 7: Under VPC, you may choose an existing VPC or the default VPC. Under the default VPC, two subnets were selected and click Save and continue.

In a few minutes, the domain will be onboarded with the status ready with the User profile name.

Tutorial 2: Launch Amazon SageMaker Studio Using Amazon Management Console

Step 1: In the search bar, type the word 'SageMaker' to open the Amazon SageMaker Console. On the left-hand side menu navigate to Studio and you will see the new domain name created for the user profile, click Open Studio.

Amazon SageMaker Studio will take a few moments to launch.

You will be directed to the Amazon SageMaker Studio homepage to confirm that you have successfully launched a SageMaker Studio Domain.

Tutorial 3: How to train and deploy an image classification model using Amazon SageMaker Jumpstart

In this tutorial, Amazon SageMaker Jumpstart will be used to train an image classification model on the Hurricane Harvey 2017 dataset.

Step 1: The dataset and Jupyter notebooks will be imported into Amazon SageMaker Studio by:

Selecting File -> New -> Terminal

The terminal environment will launch as shown below:

Step 2: Download the Hurricane Harvey 2017 dataset to the Amazon SageMaker Notebook by copying this code into the Terminal

mkdir lcnc
cd lcnc
sudo yum install -y unzip
curl 'https://static.us-east-1.prod.workshops.aws/public/40de25f9-f9de-4fba-8871-0bf4761d175e/static/resources/finserv/vision.zip' --output vision.zip
unzip vision.zip

The image files will take a few seconds to be imported.

Step 3: Open the Data pane on the left-handside menu click to view the 'lcnc' folder.

Step 4: Explore the data and copy the data from the local SageMaker folder to S3 and navigate to the lcnc/vision/ folder.

Step 5: Double-click on the jupyter notebook 'explore-data.ipynb' and click Select.

This will initiate the kernel.

Step 6: Run and execute all the cells in the jupyter notebook by pressing on your keyboard Ctrl + Enter.

The files in the local folder have been successfully uploaded into your Amazon S3 bucket.

You may also check the that the image files have been uploaded from local folder and into your Amazon S3 bucket 'damage-clf' folder.

Step 7: Click on the 'home' icon to navigate to the homepage and scroll down to the bottom to SageMaker Jumpstart and click 'Models, notebooks, solutions' to view the 'Model zoo' which includes pre-trained models, example notebooks and pre-built solutions.

Step 8: Scroll down until you reach the section 'Explore all image classification models (162)' and double-click this hyperlink.

Step 9: Select the model Resnet 50 and click View model.

Step 10: From this pre-trained 'Resnet 50' model, you may click deploy to deploy the pre-trained model (ImageNet) as a SageMaker real-time endpoint to get inference.

The deployment process will take a few seconds to complete.

The model endpoint is in service and ready to make inference.

Tutorial 4: Fine tune the model and bring in custom data

Step 1: To fine-tune the model on custom hurricane damage images, choose the S3 location of the images we just uploaded using the 'explore-data notebook' and specify the instance type we want to use for training as shown below.

Step 2: Select the instance type 'ml.c5.2xlarge'.

Step 3: Provide a model name e.g. hurricane-damage-abc

Step 4: Custom output S3 bucket

Select Default output S3 bucket.

Copy the S3 URI of the training dataset.

Step 5: Update the hyper parameters with the configuration below and select Train.

Training the model will take a few moments.

The model was successfully trained using custom data uploaded into Amazon S3.

Note: You may also inspect the training model artifacts saved in the Amazon S3 bucket.

Step 5: To deploy the fine-tuned model, configure the settings such as specifying the S3 URI path to save the model artifact.

Step 6: Under Security settings, select 'Find VPC', choose three subnets and also the security group and click Deploy.

After a few minutes, the endpoint is ready with the status'in service'.

Step 7: On the left-handside pane, double-click on the jupyter notebook 'make-predictions.ipynb'.

And execute the cells in the notebook to make predictions.

Important note: On cell 5, be sure to replace default endpoint name from screenshot below and enter the endpoint name for your deployed model from Step 6.

Step 8: Make predictions from the test data 'damaged'.

When making predictions on new data (i.e. photos), the probability is that property is 'damaged'.

Step 9: Make predictions from the test data 'not-damaged'.

When making predictions on new data (i.e. photos), the probability is that property is 'not damaged'.

Clean Up Resources

Once you have successfully deployed the model you must delete the endpoint to avoid surprise end of month bills.

Step 1: Delete the endpoints by navigating to Deployments-> endpoints

Step 2: Click on the hyperlinks to delete the model endpoint.

Conclusion

In this lesson, you have learnt how to set up Amazon SageMaker Studio for your machine learning project and also deploy an image classification model using pre-trained models in Amazon SageMaker Jumpstart. Keep on building and exploring in the 'Model Zoo' of Amazon SageMaker Jumpstart.

Until the next lesson, happy learning! 😀

References

Resources

Next Lesson

The next few lessons will delve into a mix of classic machine learning modelling techniques as well as AI.

Last week: AWS re:Inforce 2023 on 13-14 June

You may watch on Youtube the keynote from CJ Moses, Chief Information Security Officer (CISO), AWS last week. You may also watch the leadership sessions, keynotes and breakout sessions from AWS re:Inforce 2023 at this link.

Coming soon: AWS re:Invent 2023 conference

You may register now for AWS re:Invent 2023 conference on November 27 to December 1 2023 in Las Vegas.

You may watch the AWS re:Invent 2022 keynote from Amazon CEO Adam Selipsky on Youtube on-demand.