This blog is part of MSP Developer Stories initiative by Microsoft Learn Students Ambassador (India) Program - https://studentambassadors.microsoft.com/ which is aimed for student communities to Learn, Lead and Empower.
It's been around 100+ days since the first COVID-19 case in India and 60+ days into the earliest Lockdown Order by our Government. Although people understand that the extension is for their own good and to endanger the dreadly Corona Virus, social media was abuzz with reactions. Extended lockdowns are having disastrous to the economy along with psychological impact on the people. We see shift across the fields like remote learning, work from home, and transformation of business models. In order to ensure the new directions are on right move, data-driven decision making is vital and in this digital world key source of data is social media to gauge public sentiment.
This post elucidates the usage of Azure Cognitive Services Text Analytics API, an Artificial Intelligence & Machine Learning method to perform Sentiment Analysis of large unstructured data.
Case Study - Twitter Sentiment Analysis on Lockdown Extension using Microsoft Stack – Flow, Azure, Cloud, Cognitive Services & Power BI.
- Extract Twitter feeds into excel using Power Automate
- Azure Cloud & Cognitive Services to generate the key & end point for Text Analysis API
- Power BI Desktop to import and transform data
Before getting into nitty gritties of the use case, let’s have a brief of various technical concepts that will be used for implementation.
Sentiment Analysis extracts views either positive, negative or neutral from the given text which can be used for surveys, social media analyses and overview of psychological trends.
Microsoft Azure Cognitive Services is a collection of machine learning and AI algorithms in the cloud for development projects. You don’t have to be a Data Scientist to harness the power of ML. It’s as simple as calling a library, under the hood it does complicated stuff, but from use perspective it just inputs and outputs practically.
The Text Analytics API is a cloud-based service that provides advanced Natural Language Processing over raw text, and includes four main functions: sentiment analysis, key phrase extraction, language detection, and named entity recognition.
The Text Analytics client object authenticates to Azure using a key and provides functions to accept text as single string or a batch and returns a sentiment label and score of the entire input.
Microsoft Power Automate is a service that helps to create automated workflows between the apps and services to synchronize files, get notifications and collect data.
Microsoft Power BI Desktop which is a free application and can be installed on local computer that connects to, transform and visualize the data.
Now let’s start implementing the below flow:
Pre-requisites – Set up the Environment :
- Create an account with Microsoft 365 and access Power Automate App : https://us.flow.microsoft.com/en-us/
- Create an Excel with column Headers – Tweet Text, Location and DateTime and upload into OneDrive which can be accessed via logging in to Microsoft 365
- Access https://portal.azure.com/ through your Microsoft account. Ensure you have a free (Students can have a free account worth $100) or paid subscription
- Install & set up Power BI Desktop: https://powerbi.microsoft.com/en-us/desktop/
Walk-through of the implementation
Step 1 : To extract Twitter feeds into excel using Power Automate, login to Microsoft flow as mentioned in the Pre-requisites.
Create a New Automated flow and select a twitter trigger :
Mention the Search text as “Lockdown extension”. Add a New Step and Choose an action to select Excel that was uploaded in OneDrive, configure the excel / table details as below :
Save and Test the Flow, when refreshed, excel file starts getting filled automatically with the feeds from Twitter that are tagged # Lockdown extension :
Step 2 : To generate the key and endpoint for Azure Cognitive Services Text Analytics API, login to Azure portal.
Create a new resource under Cognitive Services :, Supply the Name, Subscription, Location, Service plan and Resource group
Once Deployment is completed, secure the generated Keys & Endpoint :
Step 3 : Access the Key and Enpoint generated in Step 2 to invoke and apply the function in Power BI Desktop data that was imported from excel tweet file
Launch Power BI Desktop and load the Twitter Feed excel file via Get Data :
Select GetData -> Blank Query to build the Custom Function for invoking the Cognitive Services Text Analytics API Key & Endpoint :
The Sentiment Analysis – Text Analytics function above uses a Machine Learning classification returns a score between 0 and 1, where 1 is the most positive indicating how positive the sentiment expressed in the text.
Apply the function on the Twitter Text feed column to obtain Sentiment scores.
A custom column has been added to derive the text (Positive / Negative / Neutral) based on the arrived sentiment score.
Unlike the existing process - collecting data via surveys, summarizing the emotions of people during a specific time interval, this solution depicts the emotions of the live twitter data with higher accuracy presenting the sentiment score which helps to understand people’s sentiment towards the epidemic, government decision to extend the lockdown and also helps businesses to make a 360 degree view for decision making and enablement.
Notice the Low Code pattern here in the entire exercise, we did not get into the details of the AI & ML capabilities of Azure Cognitive Services, or even Text Analytics which are pre-trained that enable to process large amount of data.
Top comments (0)