Microsoft Cognitive Services (Computer Vision)

Hello everyone, Welcome to my blog on Azure Cognitive Services. This blog is written as part of MSP Developer Stories initiative by Microsoft Student Partners (India) program.

In this post, we will discuss on one of the hottest cognitive service provided by Microsoft Azure i.e. Computer Vision. Azure Computer Vision provides us the ability to extract information from images and analyze them. It makes use of the advanced algorithms that process images based on our requirement. All we need to do is just to make an REST API call and make use of those results in our application.

There are several things we can do computer vision API. Some of them are listed below.

• Tagging Visual Features.
• Object Detection.
• OCR (Optical Character Recognition).
• Detecting faces and facial features.
• Detect Brands.
• Categorizing Images.
• Generating Thumbnails.
• Content Moderation.

Let's say we are developing an application which guides visually impaired people. Azure Computer Vision makes it easy for us by Describing the image using Analyze API , and let them experience the real world as others see. All we need to just embed this API call in our application and then organize those results to give a beautiful description of the real world.

Before we learn those things, we must need an active azure subscription to make use of Cognitive services. If you don't have one, Azure provides a 12-month free trail, Go and grab yours. If you are a student, you can get one without any credit information using Azure For Students.

Azure Computer Vision: Tagging Visual Features

In this post, we will make use of Python API provided by Microsoft Cognitive Services to leverage the awesomeness of Azure Computer Vision in our application.

Before we start, here are the list of pre-requisites:

• Azure Subscription.
• Python.
• Python Packages:
  • matplotlib.
  • requests.
  • pillow.

If you don't have one, open the command prompt in your python environment and install the packages using pip install package-name .

Hurray!! We are all set to go.

Step 1:

Create a Cognitive service resource using Azure portal.

• In the Azure Portal, Click on All Services.
• You can find AI + Machine Learning listed in the given services. Click on it.
• In the list of services you can find cognitive services. Click on it.

• If there is no existing Cognitive service, it asks you to create one. Go on and create one by clicking on Create Cognitive services.
• Now you can find a grid of services listed in the portal. Find Cognitive Services and click and see More.

• Find Computer Vision and click on Create.
• Now it redirects you to the Create Page, where you need to fill in the name of your service, your azure subscription, region, pricing tier and a Resource group.

Find a region where F0 pricing tier is available, it indicates a free service.

• Click on Create, now it validates your request and then the deployment will be initialized.
• After the deployment is complete you can go to your resource and find your cognitive service API key.

Hold on, now let's see how we can use them.

Here's the image we will be analyzing through Computer Vision Analyze API.

Step 2:

Write a Python program to make a REST API call.

Create a python file and copy the following code into your editor.

• In the first part we are importing all the required libraries into out python code.

import requests
import matplotlib.pyplot as plt
import json
from PIL import Image
from io import BytesIO

• If you are using Jupyter Notebook to run this program, you need to add the below line of code.

% matplotlib inline

• Next, in order to make an API call, we need to have our subscription key and endpoint which you can find in your deployed resource.

subscription_key = "Your-subcription-key"
endpoint = "Your-end-point"

• Initialize the subscription-key and endpoint variables with your respective key and endpoint.
• Analyze URL is the corresponding API call for Azure Analyze API.

analyze_url = endpoint + "vision/v2.1/analyze"

• Next there is the url to the image we wanted to analyze.

image_url = "https://cdn.geekwire.com/wp-content/uploads/2014/03/DSC06612.jpg"

• Then, we will make a post request to the API with the corresponding headers and the parameters.
• We can send in the image data using URL or Bytes data. We are using the url attribute to pass the image to Analyze API

headers = {'Ocp-Apim-Subscription-Key': subscription_key}
params = {'visualFeatures': 'Categories,Description'}
data = {'url': image_url}
response = requests.post(analyze_url, headers=headers,
                         params=params, json=data)
response.raise_for_status()

• The returned response from the post request contains the analysis we need in the form of json.

analysis = response.json()

• In the analysis, we have description of the image with the text attribute.

image_caption = analysis["description"]["captions"][0]["text"].capitalize()

• To add aesthetics to our output we will display the image from the url along with the caption.

• To do that, we need to get the image from url and get the image bytes and display the image using plt.imshow() method.
  

image = Image.open(BytesIO(requests.get(image_url).content))
plt.imshow(image)
plt.axis("off")
plt.title(image_caption)
plt.show()

Here we go with the description of the image we have sent to the Analyze API

That's AWESOME. Azure had made it too easy for us to analyze images using Cognitive Services. The real implementation of it involves months of training, even years to build such an awesome Deep Learning Model.

Thank you so much Microsoft for this wonderful opportunity. I would also like to thank the MSP Global Manager Pablo Veramendi, MSP India CPM's Aparna Mittal and Angela Carlson and our Microsoft AI Developer Advocate Arkodyuti Saha for this wonderful initiative of MSP Dev Stories.