DEV Community: Joshua Mangiola

A New Beginning

Joshua Mangiola — Mon, 19 Oct 2020 00:06:39 +0000

For the latest updates on this experiment click here.

About a month ago, while reflecting on recent events I asked myself a question, what's next? I've asked myself this question many times in life and at that moment I realized the path I was on wasn't going to cut it for me anymore. I needed to do something else, something new and exciting but what though? I found the answer two weeks later while browsing university courses, Data Science and more specifically Machine Learning.

Why Data Science and Machine Learning?

I have always had a broad interest in Machine Learning and Data Science, but I thought it was too complicated for me to understand and just kept to being a Software Engineer. I've decided now's the time to learn it though and I am taking the time to pursue this. I want to learn Data Science and Machine Learning as it looks like it will be a revolution to many industries, and I would like to be part of that revolution. Machine learning has the ability to make mundane tasks automated and do things we could have never thought possible only a few years ago.

My Goals

I have finally decided now is the time to learn these concepts and thus I have taken a year off to achieve. The goals I have set myself to achieve within the next year are.

Being able to understand the concepts of machine learning and the theory that sits behind it.
Have developed and released at least four projects that apply knowledge learnt to a practical problem.
Finding ways of delivering value to other people with Artificial Intelligence.
Inspiring others to try something similar or share what they are learning.

What's my plan?

I plan on doing a Masters in Information Technology with a specialization in Data Science at Deakin University mainly, but I know from experience that university education isn't enough and I need to learn with a combination of University Coursework, online courses, books and personal projects. To do this I have created a list of courses and books that I want to complete within the year. I will also combine these skills I learn into projects that will be deployed and usable.

First Six Months

In the first six months I plan to focus on laying the foundations and gaining the basic knowledge I need to progress onto more in-depth learning. The following courses, books and projects should help achieve that and of course if I find anything missing, I can add it later.

Elements of AI Course
Data Science Specialization (Coursera)
Hands on Machine Learning with Scikit-Learn and TensorFlow Part 1 (Book)
Mathematics for Machine Learning (Book)
CS50 AI Course
Project 1 (TBD)
Project 2 (TBD)

Next Six Months

After completing the first set of goals, I plan to spend the following six months working on courses, projects and books that build upon the knowledge learnt within the foundational period, these are part of that process.

All Courses by fast.ai
Some Courses by Deeplearning.ai (TBD)
Hands on Machine Learning with Scikit-Learn and TensorFlow Part 2 (Book)
Full Stack Deep Learning Course
Self-Driving Cars Specialization (Coursera)
Project 3 (TBD)
Project 4 (TBD)

How Do I Keep Myself Accountable?

I plan to keep myself accountable during this period by blogging and creating content about the entire process. I also have created a page that tracks my live status (click here to go there) on how far I'm through this experiment. I will be blogging a monthly summary on what I've learnt in the last month and the progress I've made as well as posting the projects I've created.

What's Next?

You will be hearing more from me in the future about what I'm doing during this process and I would love to hear from you in the comments on what I can do to help you or what you'd like to see, but until next time thanks for reading.

How I Passed AI-900 Azure AI Fundamentals

Joshua Mangiola — Sun, 18 Oct 2020 23:01:45 +0000

If you’ve been affected by recent events you can book an exam for the limited price of $15 USD until the 31st of December 2020 (T&Cs Here)

Recently I decided to go for the AI-100 Azure AI Microsoft certification, I stumbled upon this certificate while researching the AI-100 Certification. This certification is designed to go over the fundamentals of machine learning and artificial intelligence in Azure. AI-900 doesn’t seem to be discussed much online but I decided I was going to get AI-900 as practice for the more difficult AI-100 exam. I set myself a personal time limit of two weeks to prep for the exam as I was familiar with the Azure AI suite of products, but you may want to go for a longer timeframe.

What does the Exam Consist of?

According to Microsoft the exam consists of the following sections

Describe AI workloads and considerations (15-20%)

This section consists of Identifying the Azure service that should be used for an example workflow, you can master this easily by going through the Microsoft Learn course and keeping notes of what each Azure service does.

Describe fundamental principles of machine learning on Azure (30-35%)

For this section you will need to be able to classify different machine learning scenarios that require either regression, classification, or clustering. You will also need to understand the difference between features and labels, training and validation datasets, algorithms used in Azure ML, and how to interpret model evaluation metrics returned in Azure ML. Some other topics you will need to understand are how to ingest data, prepare data, complete feature selections, feature engineering, train, evaluate models, and deploy and consume models in the Azure ML Studio. Also, you will need to be able to explain Automated ML in Azure ML Studio.

Describe features of computer vision workloads on Azure (15-20%)

To pass this section you will need to be able to describe the use cases for Azure Computer Vision and determine the computer vision solution needed such as image classification, object detection, semantic segmentation. OCR (Optical Character Recognition) and facial recognition. The Exam also requires you to understand the difference between Computer Vision, Custom Vision, Face, and the Form Recognizer service.

Describe features of Natural Language Processing (NLP) workloads on Azure (15-20%)

For this section to achieve a pass you need to be able to identify the workloads needed for different NLP scenarios. These include key phrase extraction, entity recognition, sentiment analysis, language modelling, translation, speech recognition, and speech synthesis. Other requirements are being able to understand each service (Text Analytics, LUIS (Language Understanding Intelligence Service, Speech, and Text Translator Service).

Describe features of conversational AI workloads on Azure (15-20%)

This section is about the conversational AI services that Azure offers, these include QnA Maker and the Bot Framework. You will need to know how the services work and use cases for each of the services such as a web chat bot or voice assistant.

Resources I used to Learn

There doesn't seem to be many resources online for the Azure AI-900 exam but the Microsoft Learn path (https://docs.microsoft.com/en-us/learn/certifications/exams/ai-900) is great and covers almost everything you will need to learn for the exam. Other than that, I would recommend signing up for the free Azure Trial or Azure for Student (Check out this blog for more details) and playing around with all the services to get experienced with them. There are some other resources such as this one that goes over the exam in a lot of detail (https://stanislas.io/2020/07/01/preparation-guide-for-ai-900-microsoft-azure-ai-fundamentals-certification/).

My Experience of the Exam

I took the Exam online due to the current situation and thus had to use a laptop that had a webcam so the proctor would be able to monitor me. The process is simple enough, you follow the prompts, install the proctor software, and go through the verification process. Once I completed the verification the proctor released the exam, I noticed it consisted of mostly multiple-choice questions with a few drag and drop and finish the sentence questions thrown in too. I found the questions easy to understand and I completed the test within 15 minutes of starting. Once I submitted my results, I got the score of 936/1000 which I was pleased with.

Summary

Overall, I think the AI-900 Exam is pretty basic but if you can get it for a discounted price it becomes a good starting block on your journey to achieving the more useful Azure AI-100 certification. By completing the Microsoft Learn course and just playing around with the services while taking notes you should be able to pass with ease.

Getting Started with Azure Content Moderator

Joshua Mangiola — Sun, 18 Oct 2020 22:38:57 +0000

Azure Content Moderator is an Azure Service that uses machine learning to help with moderation for different forms of content such as text, video, and images. The service provides both a Moderation API and Review API, it’s something interesting to learn about as this service is needed to pass the AI-100 Exam. In this post I will go over what you can do in the content moderator and how to implement it using the Python library.

Text Moderation

Text moderation is used for many different types of applications, some examples include Chatbots, Discussion Forums and Document Verification. The Azure Text Moderation API allows for text to be supplied and it will return a list of potentially harmful words, the type of harmful words, and if any personally identifiable information is in the text.

Creating a Content Moderator in Azure

First you will need to open the Azure Portal (portal.azure.com), once it has loaded click on the search bar at the top and type in "Content Moderator" and click on the Marketplace Item. You will then need to select your subscription, resource group, region, pricing tier (choose Free F0 for testing), and enter a name (this name must be unique). Next hit Review and Create and if it passes validation hit create and the content moderator will deploy which should complete within a few seconds to minutes.

Now that it's deployed you can check out the following page to test the API. You will need to click on your region of choice (Australia East in my case) and then scroll down to the field Ocp-Apim-Subscription-Key and paste your key from your Azure Content Moderator Service (Available under the "Keys and Endpoints" tab). Hitting send now should give you a response like below that has three major sections which are PII (Personally identifiable information), Classification, and Terms, we will go over each section in more detail below.

Example of an API Response

Pragma: no-cache
apim-request-id: 53ee7e6e-fc5a-4c78-9170-26507c5a6***
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
CSP-Billing-Usage: CognitiveServices.ContentModerator.Transaction=1
Cache-Control: no-cache
Date: Fri, 16 Oct 2020 00:53:21 GMT
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Content-Length: 1085
Content-Type: application/json; charset=utf-8
Expires: -1

{
  "OriginalText": "Is this a cr*p a** email abcdef@abcd.com, phone: 6657789887, IP: 255.255.255.255, 1 Microsoft Way, Redmond, WA 98052",
  "NormalizedText": "   cr*p a** email abcdef@abcd.com, phone: 6657789887, IP: 255.255.255.255, 1 Microsoft Way, Redmond, WA 98052",
  "Misrepresentation": null,
  "PII": {
    "Email": [{
      "Detected": "abcdef@abcd.com",
      "SubType": "Regular",
      "Text": "abcdef@abcd.com",
      "Index": 25
    }],
    "IPA": [{
      "SubType": "IPV4",
      "Text": "255.255.255.255",
      "Index": 65
    }],
    "Phone": [{
      "CountryCode": "US",
      "Text": "6657789887",
      "Index": 49
    }],
    "Address": [{
      "Text": "1 Microsoft Way, Redmond, WA 98052",
      "Index": 82
    }],
    "SSN": []
  },
  "Classification": {
    "ReviewRecommended": true,
    "Category1": {
      "Score": 0.0011853053001686931
    },
    "Category2": {
      "Score": 0.49122610688209534
    },
    "Category3": {
      "Score": 0.98799997568130493
    }
  },
  "Language": "eng",
  "Terms": [{
    "Index": 3,
    "OriginalIndex": 10,
    "ListId": 0,
    "Term": "cr*p"
  }, {
    "Index": 8,
    "OriginalIndex": 15,
    "ListId": 0,
    "Term": "a**"
  }],
  "Status": {
    "Code": 3000,
    "Description": "OK",
    "Exception": null
  },
  "TrackingId": "AUE_ibiza_677e548d-ccf1-475f-b0ed-3a1fa1fd2137_ContentModerator.F0_78430acc-0452-45c9-bf7f-1e45b5651***"
}

Personally identifiable information

The Content Moderation API can detect when personal information is present in the text sent to the API, some of the fields it can detect are email addresses, mailing addresses, IP addresses, phone numbers and social security numbers. When one of these fields are detected the Content Moderation service returns a new PII object in the JSON response like below. The object contains what type of field it is and the location of the field to censor or remove it in the text.

"PII": {
    "Email": [{
      "Detected": "abcdef@abcd.com",
      "SubType": "Regular",
      "Text": "abcdef@abcd.com",
      "Index": 25
    }],
    "IPA": [{
      "SubType": "IPV4",
      "Text": "255.255.255.255",
      "Index": 65
    }],
    "Phone": [{
      "CountryCode": "US",
      "Text": "6657789887",
      "Index": 49
    }],
    "Address": [{
      "Text": "1 Microsoft Way, Redmond, WA 98052",
      "Index": 82
    }],
    "SSN": []
}

Classification

In the JSON response returned by the Content Moderation API there is a classification field which recommends if the text should be reviewed and then scores it on each of the following categories.

Category 1: Potential presence of language that might be considered sexually explicit or adult in certain situations.
Category 2: Potential presence of language that might be considered sexually suggestive or mature in certain situations.
Category 3: Potential presence of language that might be considered offensive in certain situations.

"Classification": {
    "ReviewRecommended": true,
    "Category1": {
      "Score": 0.0011853053001686931
    },
    "Category2": {
      "Score": 0.49122610688209534
    },
    "Category3": {
      "Score": 0.98799997568130493
    }
}

Terms

When passing text to the Content Moderation API, any word that potentially contains profanity is returned as a Term object in the JSON response. The term object contains an Index value to locate the text, the term detected, and the ID of the list being used. This will not be zero if you are using a custom list of words.

"Terms": [{
    "Index": 3,
    "OriginalIndex": 10,
    "ListId": 0,
    "Term": "cr*p"
  }, {
    "Index": 8,
    "OriginalIndex": 15,
    "ListId": 0,
    "Term": "a**"
  }]

Image Moderation

Image Moderation in Azure Content Moderator works similarly to Text Moderation, but it's used instead to analyse images for adult and racy content, detect text with Optical Character Recognition and Detect Faces. In this post I will only go over detecting adult and racy content within images, but you can play around with the other uses from within the Image Moderation API Console.

Using the Image Moderation API Console

The easiest way to test the Image Moderation functions of the Content Moderator is to use the API Console, you can access the console by using the following link. In the console you will need to select your region then enter your subscription key from the Azure portal in the Ocp-Apim-Subscription-Key field. Once that's done you can change the request body from the default value to the image of your choice and then hit send and you should get a JSON response like the one below returned.

Example API Response

Pragma: no-cache
apim-request-id: 576338ac-132e-4e22-89f8-df8056ce05af
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
x-content-type-options: nosniff
CSP-Billing-Usage: CognitiveServices.ContentModerator.Transaction=1
Cache-Control: no-cache
Date: Fri, 16 Oct 2020 04:58:20 GMT
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Content-Length: 458
Content-Type: application/json; charset=utf-8
Expires: -1
{
  "AdultClassificationScore": 0.021854337304830551,
  "IsImageAdultClassified": false,
  "RacyClassificationScore": 0.045791342854499817,
  "IsImageRacyClassified": false,
  "Result": false,
  "AdvancedInfo": [{
    "Key": "ImageDownloadTimeInMs",
    "Value": "1586"
  }, {
    "Key": "ImageSizeInBytes",
    "Value": "273405"
  }],
  "Status": {
    "Code": 3000,
    "Description": "OK",
    "Exception": null
  },
  "TrackingId": "AUE_ibiza_677e548d-ccf1-475f-b0ed-3a1fa1fd2137_ContentModerator.F0_2fbb5e6f-c7ab-4143-a16e-cb18c7511621"
}

When looking at the Example API response we can see that the main fields are the following.

AdultClassificationScore - This value ranges from 0-1 and represents if there could potentially be any content in the image that may be considered adult or explicit in certain scenarios.
IsImageAdultClassified - This uses internal thresholds on the AdultClassificationScore to convert the result into a Boolean value
RacyClassificationScore - This value ranges from 0-1 and represents if there is any racy content contained within the image.
IsImageRacyClassified - This uses internal thresholds on the RacyClassificationScore to convert the result into a Boolean value

The Image Moderation API can also detect text with OCR, Detect Faces and Create custom list, I may go through these functionalities in a future post.

Using the API in a Python Application

If you want to follow along check out this Google Colab workbook I've created that guides you through the python code required to use the Content Moderator API.

The code snippet below uses the Content Moderator Python API to analyse a text file with the Content Moderation API, once that's complete it prints the response to the console. When using the snippet you will need to create a text file named azure_cm_text.txt in the ./text_files/ folder then replace the CONTENT_MODERATOR_ENDPOINT and CONTENT_MODERATOR_SUBSCRIPTION_KEY variables with the corresponding values from your Azure Content Moderator (Available under the "Keys and Endpoints" tab).

import os.path
from pprint import pprint
import time
from io import BytesIO
from random import random
import uuid

%pip install azure-cognitiveservices-vision-contentmoderator

from azure.cognitiveservices.vision.contentmoderator import ContentModeratorClient
import azure.cognitiveservices.vision.contentmoderator.models
from msrest.authentication import CognitiveServicesCredentials

CONTENT_MODERATOR_ENDPOINT = "CONTENT_MODERATOR_ENDPOINT"
CONTENT_MODERATOR_SUBSCRIPTION_KEY = "CONTENT_MODERATOR_SUBSCRIPTION_KEY"

client = ContentModeratorClient(
    endpoint=CONTENT_MODERATOR_ENDPOINT,
    credentials=CognitiveServicesCredentials(CONTENT_MODERATOR_SUBSCRIPTION_KEY)
)

with open(os.path.join('./text_files/', 'azure_cm_text.txt'), "rb") as text_fd:
    screen = client.text_moderation.screen_text(
        text_content_type="text/plain",
        text_content=text_fd,
        language="eng",
        autocorrect=True,
        pii=True
    )
    pprint(screen.as_dict())

For more details on the Python API and the C# and Java API check out this Microsoft documentation page

Overall, this post has shown that the Azure Content Moderation Service API is extremely powerful and is a great solution for content moderation, the main concepts we went through in this post are Text Moderation and Image Moderation. In a future post I will go through the Video Moderation and create a project with the Video Moderation API.

If you enjoyed this post or found it helpful feel free to leave a comment below or react to the post.