DEV Community: idiWork

Brand presence

idiWork — Mon, 29 Mar 2021 11:45:42 +0000

About

Companies and brands have always invested in having a presence; either by appearing in certain ads or through influencers on social media.

However, this investment is useless if there is no traceability of where the brand has appeared, for how long, for how many people and, most importantly, what impact and return on investment we have had on our business; as well as answering other crucial questions.

Idea

Providing traceability of our brand presence is a task that involves a lot of effort due to the volume of data processing that it requires. A successful analysis of this data will help to carry out market researches and correlations to know where to invest in future campaigns to maximize our business objectives. The objective of this experiment is to be able to exploit this huge amount of information that is generated from different data sources and its subsequent analysis for decision-making.

Thanks to artificial intelligence and parallel data processing, this is now possible.

Process

First, we must bring together all the data sources where our brand appears (TV channels, radio, social networks, Twitch, YouTube channels, etc.). Several processes in the Azure cloud will be in charge of carrying it out. Subsequently, thanks to parallel data processing and sophisticated Artificial Intelligence systems from both Cognitive Services and ad-hoc trained neural networks (all this also put into production in Azure), our brand will be able to be identified. In addition to this, the current audience level, as well as other relevant information, will be extracted at all times. Then all this information will be stored in a corporate Data Warehouse after an ETL process that will bring together all the information. Finally, an analysis will be carried out in Power BI to be able to transform all this information into knowledge that can be exploited by the analysts for decision making.

Utility

Obtaining brand presence is a good practice where a large part of capital is invested. Providing traceability and being able to analyze the impact it has is a difficult task if it’s not done with the right tools.

Thanks to Artificial Intelligence and Big Data techniques, we offer to bring all this knowledge to analysts in order to answer all their questions, maximize return on investment (ROI), as well as achieve business objectives.

Introduction

Companies have always struggled to have presence in the sector. Each year, companies invest huge amounts of money to appear in strategic locations and to be able to achieve a greater volume of sales, customers and what–nots. But, do they conduct studies and traceability of:

• Where has my brand appeared?

• For how long?

• How many people have seen my brand?

• In which medias?

• What impact has it had?

There is no doubt about the repercussion that having a presence in these strategic places provides. However, if we enrich it with a posteriori studies that are capable of providing us with information for decision-making, our return on investment (ROI) is guaranteed.

This problem creates real headaches when having to deal with large data from different sources and their subsequent analysis. But, thanks to artificial intelligence and parallel data processing, this is possible.

Architecture

The high-level architecture capable of satisfying the earlier mentioned needs would be the following:

Roughly speaking, what we would have would be a series of origins from which we will analyze when the brand is present and how many viewers it is reaching. Among these origins, we can find:

• Twitch platform

• YouTube

• Sports such as football, tennis, MotoGP, basketball…

• TVchannels

• Radio

• Socialnetworks like Instagram, Facebook…

These are just a few examples of the wide range of possibilities that we have, being able to add as many as we want. Let’s see it a little more in detail:

We would start from the base of having a number of services in Python that would be in charge of capturing and connecting to the different APIs and sources. In addition to connecting, they would make the necessary cuts to keep the data of interest.

This data would then be sent to both Azure Cognitive Services and custom AI models that will be the responsibles for identifying the brand in each of the data sources.

Finally, all this information will travel through Azure Data Factory where the Business Intelligence’s own Extract, Transform and Load (ETL) process will take place and this information will be transformed, analyzed and joined to the other data in our central data unit constituted by the Data Warehouse.

This last part will be where we nurture the subsequent analyses performed with PowerBI that will be what we make available to the corresponding department for decision making.

Preparation

The sources from which we are going to extract the information are totally heterogeneous, which forces us to carry out a set of techniques and scripts to capture the data that we need. In this case, we decided to use Python programming language, which offers a suite of tools capable of executing this heavy task.

For different social networks we have at our disposal the corresponding API to obtain the desired information (publications, likes, visits, etc.). And, both from Twitch and the different television channels, we have direct access to the signal.

Later and thanks to artificial intelligence, we will be able to detect if the brand is present, like in the next images:

Services

Once the information has been collected, different services will be in charge of detecting the presence of the brands in each image. Apart from customized models that will improve and enrich our system, we will use Computer Vision, which is one of Azure’s Cognitive Services:

Analysis of the information

Once we have the architecture assembled and the data has been processed by the Cognitive Services, the next step is extracting information from the pre-processed data to make it available to the analysis departments, which will use it for decision making.

To tackle this last part, we have decided to make use of a great ally such as Power BI.

Understanding the information

Next, we will explain how the Power BI’s information is structured.

At the top we find the sources:

• TV channels (for example)
o Tele 5.

And within each channel we can find different categories, like:

• Programs
o The comedy club => We detected a brand on:
 the stage
 the comedian’s clothes
-> We identify the comedians and detect the number of spectators

• Sports
o Football matches => We detect a brand on:
 the stadium
 the players´ clothing
-> We identify those players and detect number of spectators

• Ads (again, same process)

In Social networks:

• Instagram
o FC Barcelona account
 Publications: We detect the brands
-> We identify the player and identify the number of visits and likes

• Twitch: Same concept as the previous ones. We would be able to navigate the different streamers, identify them and detect how long a brand has been seen on the screen and for how many viewers.

• And so on…

Analysis

“Impact” is the effect caused by each of the elements. For example, the impact on a football broadcast would be calculated as the product between the seconds that a brand appears times the number of spectators. However, in a social media post the impact will be equal to its reached viewers.

The “impact” metric is something totally customizable, and must be calculated as the experts in the business dictates, because each company behaves in a different way.

Let’s start with the report. In it, the different elements will appear numbered to facilitate their reading and monitoring.

Global metrics – tab

The percentage of impact of each social network. We can see how Instagram and Facebook predominate, the first being the winner.
Percentage of impact by subtype. In this case, we can see how eSports and entertainment stand out from the rest.
Regarding television channels, Cuatro and La Sexta are the superiors.
There is no doubt about the impact that Twitch is gaining. That is why we thought it would be interesting to compare it with the impact of both social networks and the TV. In this case, we can see how Twitch could practically be put at the same level than the other two, so the investment made in this platform is being truly effective.
Finally, in terms of general impact by source (without breaking down by subtype), Twitch would be the one performing most satisfactorily.

Social media – tab

In this tab we are going to enter into a study between the different social networks:

Number of total publications vs the target set. We set a goal of 1000 and we are 67 publications above.
Although the number of publications has been exceeded, we have not received as many “likes” as expected.
However, our custom metric that relates both “likes” and visits to publications tell us that we have reached a lot more people than expected, so the objectives are being achieved.
In this chart we can see the impact of our publications by date. February was clearly the busiest month.
One of the studies to be carried out is when (time) our publications have the greatest impact. Thanks to this visual we can see how from 8 to 9 and from 13 to 17 is when we have a greater impact.
Here we can see the impact of each of the social networks. In addition, it also works as a filter.
If we want to know which person in the publication has the greatest impact, this chart will tell it. Here Cristiano Ronaldo and Lionel Messi meet tied at the top of the list. However, Iker Casillas seems to be the one with that least impact creates.
We also have at our disposal a couple of filters to filter by “Service”, this means, by a social network account in particular or by a particular person.
Finally, we can see the number of publications by type (image vs video).

If we want to filter by a specific account (FC Barcelona, for example) this would be what we would see:

And here an example of FC Real Madrid:

As the analysis shows us that the posts with Cristiano Ronaldo or Lionel Messi have most impact, the decision-makers interest would be in having these two appearing in most of the posts in order to generate even more impact. And as the publications on the FC Barcelona account work very well between 8 and 9, the account should try to publish at that specific time, or between 1 and 3 o’clock, which is a good time to publish for both accounts.

Twitch – tab

As we have seen before, Twitch is one of the leading platforms at the moment. And to many, the new television.

Personalized impact vs the objective set.
Seconds that appear on the screen vs the target.
Number of people we have reached vs the target.
When the post has had most impoct. In social networks, it is crucial to know the best time slot to publish, so this chart is very important.
Impact per person. In addition, it also works as a filter. We can see how Ibai is the leader, followed by AuronPlay.
The impact by date.

“TV Channel” tab

This tab shows the analysis of different television channels and programs:

As in the previous tabs, here we can see the customized impact vs the objective.
Total seconds vs the goal.
People we have reached vs the objective.
Time period in which the most impact has been achieved, this being mainly between 6:00 p.m. and 3:00 a.m.
Impact by service. As we can see, the ads have greater impact than a football game. But, we have to keep in mind, that the costs for appearing in an add are different for the costs to appear in a stadium or on the players’ shirt, and that, in this case, we should also elaborate metrics such as impact vs. invested money.
The impact in different TV channels.
Impact per person who appears on the screen with the brand. This info can be used in future ads and choose one person over another to appear in the ads, in order to achieve a greater impact.

If you want to read more go to this post.

Stay up to date: idiwork.com

Analysis of hot spots in physical stores

idiWork — Wed, 03 Feb 2021 09:39:39 +0000

About

There are countless processes in a physical store that produce data that can be used and exploited to generate or increase sales processes.

In addition to measuring the time spent in the store and register whether the visitor ends up buying something or not, the purpose of experiment #205 is to use facial recognition services to locate hot spots in the store, sentiment analysis to analyze the level of satisfaction of the client, and Cognitive Services to analyze bags or objects from other brands.

With all this information it’s possible to calculate a predictive stock to automatically supply the store.

Idea

The idea arises from the need and the lack of use of the information that can be exploited in physical stores. Nowadays data-driven decision making is crucial for the proper development of a business and thanks to AI this reality can be possible.

Process

First, we will use the store’s cameras to analyze the images. Then, we will develop a software that collects all the information and sends it to the cloud to be analyzed in the Cognitive Services. Once in the cloud, all the necessary analysis, the entire ETL process and a further integration with the corporate DataWarehouse will be carried out and utilized through PowerBI.

Utility

Find correlations between the average time spent in a store and the probability that a purchase happens, detect heat sources and patterns for possible rearrangements of the layout, predict stocks, analyse customer sentiment to identify possible dissatisfaction and detect brands that customers consume for segmentation.

If you want to read more go to this post.

Introduction

For many people, artificial intelligence is something reserved. And for few, a future that is yet to come. But the reality is different, since we can see around us a huge number of examples where artificial intelligence is part of our daily lives. Who has not seen a series in Netflix or booked a hotel room in Booking suggested by the product recommender? Or even used Google to search the internet.

On the other hand, as subject matter experts, we are used to companies and large organizations that do not take full advantage of their data, since data is, without a doubt, ‘the new oil’. Extracting data and making the most out of it is a complex task but, if done in the right way, the return on investment is more than guaranteed.

Today we will focus on unraveling a system capable of tracing the patterns that people follow at a shopping center; analyzing their feelings at the arrival and departure, whether they go to the fitting room, buy something and much more. This procedure is carried out with Azure and the results are shown in Power BI.

Description of the system

Data from the shopping center`s security cameras are rarely exploited. Thanks to advances in technology, we can study our consumers, not only when they interact through our e-commerce platforms but also in the shops.

The system will answer the following questions:

How many people have entered the shop?
How long have they stayed inside?
Which departments has the person visited?
How much time has the person spent in each department?
Has the person been in the fitting room?
Has the person bought anything?
How long has the person been queuing to pay?
What is the person’s feeling (positive/negative, maybe surprise at a price, etc.) throughout the visit to the shop?
Which parts of the store are most often visited?
Which bags does the person have?

Architecture

The simplified architecture to meet all the needs and answer the previous questions would be this one:

We would start from the base of having n number of shops. Each shop would have several security cameras that will be used to trace the patterns and routes of each person.

At the shop’s closing time (although it could also be done in real time), all the data stored by the security cameras would be sent to Azure’s services for further analysis.

Let’s see it in detail:

Each shop will have a database where the different data generated throughout the day will be stored. At the end of the day, an automatic process will send the images captured by the cameras to be analyzed in Azure. In order to meet the business needs, the Face and Computer Vision tools will be used to answer all the target questions that were raised before.

Finally, all the data generated by Azure Cognitive Services will be stored in an Azure SQL Server in PaaS, for later analysis in Power BI and, thus, making it available for the analysis department.

Organization of each shop

Each store will need to be divided into n number of areas so that we can draw a later analysis focused on the desired direction.

For example, in a shop like this:

The cameras would need to be strategically placed in order to visualize the shop’s:

Aisles
Entry and exit of the shop
Entry and exit of the fitting rooms
Cash desks

In addition, the shop needs to be also divided into different areas, for example, in red, green and blue zones, that may correspond to men’s, women’s and children’s departments. Like this:

Services

As discussed in the above, we will use Microsoft Cognitive Services for image processing:

Within the wide spectrum it offers us, we will use the Facial Recognition and Computer Vision services.

Facial Recognition

The Facial Recognition service detects the faces of the people who pass through the space we are studying. In addition to identifying them and being able to differentiate them from each other, it also recognizes the facial expressions and provides us the emotion the person is feeling at that specific moment.

Computer Vision

The Computer Vision service identifies the brands of the bags that our customers carry with them:

In addition, the Computer Vision service locates the people in our area, giving us the x and y coordinates of the establishment in order to identify the patterns, hot spots and much more.

Assembling the system

Now that we know what the final goal is and the parts that compose it, we’re going to try to assemble the whole system so that it’s able to function as a whole and can provide us the information we need to analyze our business so we can gain a competitive advantage:

As discussed earlier, the first step is to have each of the stores divided into sections that we are interested in studying. In addition to this, we would conduct a study to identify how many cameras would be needed and where they should be located to be able to control the entire space.
Once this is done, the next step would be to have a system in charge of collecting the photos of each of the cameras along with information like the time, camera from which it was taken, and other metadata that interests us. All the information will be stored in a database (for example, in an instance of SQL Server) that will be available in each store.
At the end of each day, there must be a process that will take all the information generated on that day and send it to our services in Azure so we can identify faces, emotions, brands, location in the store, etc.
Finally, all the information generated by the services will be stored in a centralized database (on an Azure SQL Server instance in PaaS) in order to perform the relevant analyses.

With the entry of the new GDPR and to ensure that we don’t send images over the network with customers who haven’t given their explicit consent, we may slightly change the previous assembly:

The first step is the same. We will have each store divided into sections and we would conduct a study to identify how many cameras would be needed and where they should be located.
We would need exactly the same system to collect the photos of each of the cameras along with the information about the time and camera from which it was taken and other metadata that interests us. All this information will be stored in the database that will be available in each store. This point would remain intact in both methods.
At the end of each day, there will be a process that will take all the information generated on that day and process it on the same store system, so we can bring the models of the three Azure services mentioned earlier to process the images as an IoT system in the Edge. With this technique the faces will be stored as a unique, irreversible, and anonymous identifier (for example, as a hash).
Once we have processed the images, the information will be stored in the same database where the images and raw data have been stored.
At this point we could delete all raw information so that we would only have anonymous data. What’s more, we could make a third approach that would be to process the images in real time, not having to store any raw images and eliminate any possibility of having any privacy-related problems.
Finally, once we have stored all the processing information in the database, we will run one last process before the analysis which is an ETL (Extract, Transform and Load) of Business Intelligence to gather all the processed data from all the stores into a centralized database in Azure (in an Azure SQL Server).

Analysis of the information

Once we have assembled the architecture and processed the data using the Cognitive Services,the next step is crucial; we must extract information of that preprocessed data in order to bring it to the knowledge of analysts, who will use it for decision-making.

To address this last phase, we’ve decided to make use of a great ally like PowerBI. The report is as follows:

We have numbered it in such a way that it’s easier to explain and locate the items in the report.

A bar chart in which we see the percentage of the brands of the bags that the customers carried. In this case almost 80% were Zara's.
We can set different target values. In this case, we have established as our target that 50% of people who enter the store should buy an item. In this example 51.61% of the people ended up buying something.
In addition to the sales objectives, we have also observed that there may be a correlation between the purchase and the visit to the fitting room. That’s why we established as a target that at least 40% of customers should go through the fitting room. In our case, 36.11% have done so; we’re a little below the target.
We are also interested in having positive feelings within our store. One of the moments where most discontentment occurs is while waiting. So to do this, we have set a limit of 10 minutes of queue. On average, our customers spent 12.24 minutes queuing, a value somewhat above our goal.
On the other hand, another aspect that also has a correlation with the final purchase is the time spent in the store. We have set 15 minutes as the minimum time a person should spend in the store. In this case, the average time spent in the store was 20.67 minutes, so this target was achieved.
We also have some gender statistics. In our store 60% of the visitors are women and 40% are men.
As for age, we can see how men and women are, on average, 34 and 44 years old respectively.
As for the areas we defined earlier, we can see the percentage of people who passed through the different areas to take a look at the products. In the case of zone 1, 78% of the people who entered the store ended up passing through zone 1.
In the case of zone 2, 60%
Finally, in the case of zone 3, only 28% ended up visiting the zone. This may indicate that the area is not well designed, signalized or just doesn’t have very attractive products.
In this bar chart we are interested in seeing the total number of people who entered the store each day, and the predominated feeling during the day. This can help us detect potential key days when more staff or better organization is needed. In addition to this, also observe peaks (both high and low) of visits.
Finally, a filter to segment by date.

In point 11, we see that there is a day with a predominantly negative feeling. We’re talking about January 31:

When we filter it we can draw numerous conclusions:

As we have said before, the first thing is that the prevailing feeling on this day is negative.
We can see how the percentage of purchases is 68.3%, well above average (51.61%)
The visits to the fitting rooms have reduced to 31.3%. This can happen because we are having a special Sales day, where it comes out more profitable for customers to buy and then return than waiting in line to get to the fitting room.
Finally, the average time spent queuing has increased dramatically to 42.3 minutes

Sentiment and pattern analysis

An example of a successful purchase, without any negative emotions or highlights:

In this chart we can see the path that the person does within our establishment. We can also see where the person is and how much time he/she spends in the same place. This chart also shows the emotions the visitor is having throughout its stay. We see how in one of the points it reflects surprise; probably because the price of something he/she likes is lower than his/her expectation. Also, after leaving the fitting room the visitor seems happy with the result.
Slicer in charge of making the timeline move forward so that you can see chart 1 dynamically, making it possible to visualize the person’s journey throughout the store as if they did it for real.
We can also see the actions this person has taken. In this case, the visitor made a purchase (the garment that surprised) and a return.
The brands on the bags with which the visitor entered
Gender data
Age of the visitor
The time spent in the store. In this case it has been higher than the minimum of 15 minutes that we set before
The time spent in the queue, this being 8 minutes, less than our maximum.
And finally, we have at our disposal a slicer for filters per person.

Next, let’s look at an example of a negative situation in the establishment:

This visitor, like the previous one, found a garment that was intriguing and, after passing through the fitting room, was convinced enough to buy it.

However, we can see how at the end she didn’t take any action. If we look closely at the pivot chart, just as she was queuing her feelings changed drastically first to anger and finally to disgust.

And we can clearly see how this person was in the establishment for 32 minutes, and 24 of them she passed waiting in line. A time period that is unrealistic to leave satisfied customers or create pleasant emotions in the establishment.

These types of reports are very close to those of traditional Business Intelligence, where we have graphs, metrics, KPIs,etc. to analyze our business. We can add as many as our analysis and business team needs to make the right decision. We have opted for these as an example, but there are countless studies and metrics to calculate the fruit of the idiosyncrasies of each establishment.

Benefits

Different KPIs for sales and stock
Sales Forecast
Intelligent detection of areas of improvement
Suggestions of promotions
Seller analysis
Customer behavior analysis

Other cases of use

We have adapted this experiment to the retail industry as we - think it’s a case in which it could be applied perfectly in order to get great results and a great return on investment.

However, there are numerous examples where this system could also be implemented:

Sports

Track athletes, their relative position and movements they make when scoring. Analyze correlations and build the key to success.

In addition, we could also analyze the audience that attend these events, their patterns, behavior, etc.

Medicine

We know that medicine is one of the sectors where AI is very present. This system could provide us with disease control, tracking of people with the disease and much more. This could help to prevent and/or combat diseases in a much more effective way.

Farming

Although it may seem like a very conservative sector, the use of new technologies is the tendency. The competitive advantages that can be achieved through its inclusion are considerable.
This system would allow us to study and control how animals move,what they eat, whether they have drunk enough water or not, and if they act accordingly.

Transportation

In the transport sector this system also can have a lot of use. In any space where there is movement of people, vehicles or objects it will allow us to analyze all the information completely autonomously and draw conclusions.

Traceability of people in department stores, where there are a large number of stores
Traceability at airports to maximize their economic performance through passenger purchases
Covid-19 traceability
Optimization of activities (both lucrative and leisure) on cruise ships
Optimization and time waiting in line
Traceability in supermarkets

If you want to read more go to this post.

Facial Recognition Access Control

idiWork — Tue, 28 Jan 2020 11:24:28 +0000

About

The Experiment #103 researches about the possibilities of applying Cognitive Services to create an access control system only managed by an Artificial Intelligence by giving it the capabilities of speaking human languages and recognizing human faces.

Idea

The project tries to simulate an automatic access control system with facial recognition. The application will tell the visitor if he/she is or is not in a white list in order to open and welcome him/her or to keep the door closed to strangers. The idea is based on an old project from huckster.io created by Ethan Kusters and Masato Sudo and published in 2016.

Utility

For this simulation we will build a smart box. This box will contain an item hidden behind a little door. The door will be locked until the application recognizes a face included in the white list. If the Artificial Intelligence recognizes the visitor the door will automatically open, showing up the item. This way, the owners of this system only have to take care of who take part of the white list by enrolling the different visitors.

Process

First, we are going to create a Windows Universal Platform application with C Sharp and .Net Core. This application will be our user interface. After that, we will connect a Webcam to interact with the Cognitive Service Vision Face API. Then, we will to ensemble all the electronic parts, a LED light, a button and a servo motor. We will install Windows 10 IoT Core on a Raspberry Pi 2 model B GPIO device to run the application on it. Also, we will build our 3D printed smart box to contain the entire project and the electronics. Finally, we will plug in a speaker to hear the machine speaking thanks to the Cognitive Service Text to Speech API.

Advantages

The principal advantage of using Cognitive Services in this simulation is the possibility of identifying people quickly and accurately. The Artificial Intelligence can work for us 24 hours a day, 7 days a week without any interruption. This way a company will save money and resources and will be very much efficient controlling the accesses to a building for example. In addition, with the capability to speak we can give the machine a more human appearance.

Architecture

First, we created a Windows Universal Application to run the project on desktop and mobile. This app allows us to register the faces of our whitelisted users (01). These people will be the ones with access permissions.

The app connects to Azure (02) and its Cognitive Services (03) to use the Text to Speech (04) service and the Face (05) service we have previously deployed.

With the Face cognitive service, we can store all users by adding their photos to a Person Group List (06) in the cloud.

There are reverse arrows in the diagram because the process works also for consulting the Person Group list and tell us if the visitor is registered on it.

If you want to read more go to this post.

Step by Step

1. How to work with Face Cognitive Service

First, we are going to create the Face Resource from our Azure portal under Cognitive Services group:

If you want to see the whole process just go to this post.

2. How to set up the IoT device hardware

We will use a Raspberry Pi 2 Model B to run our IoT project. Also, we need to attach to it some peripherals like: Nano USB Wi-Fi Adapter, USB Compact Speaker Set and Microsoft LifeCam HD-3000.

If you want to see the whole process visit this post.

3. How to pack and launch the IoT project

The 3D model of the project box was created in SketchUp. Inside the six faces, there are individual spaces for each component.

Then, we used a Slicer Software to convert the STL 3D files into G-Code that our 3D Printer can understand and print.

Finally, we printed the 3D parts in a Prusa i3 MK2S printer with black, red and white filaments from Smart Materials.

If you want to see the whole process just go to this post.

Links

Check the code in the GitHub repo!

Create a hotel customer reviews classification with Azure

idiWork — Tue, 07 Jan 2020 07:59:26 +0000

About

The Experiment #102 researches about the possibilities of applying Cognitive Services and Deep Learning to understand and manage a huge amount of information from human interactions without the human intervention. We will discover if the Artificial Intelligence is capable of collect all this raw data, classify it and manage all the different situations.

Idea

The project tries to simulate an automatic classification of comments or reviews in a business environment. Any company wants to identify critical requests as soon as possible to try to solve it before them become a big problem, regardless of the number of messages that arrive, or the language in which them were wrote, or the time that them were sent.

Utility

For this simulation it is been chosen the case of an hotel. We will imagine that the customers have access to a mobile application in which they can leave their comments, complaints and suggestions. They can use images to illustrate their point of view or location and text messages to explain it. Due to content, the reviews will be classified as good, neutral or bad.

In this case we want to classify each comment and identify the bad, urgent and important ones to manage them quickly and, if it were necessary, report to a different human agent depends of the location (room, bar, restaurant, swimming pool, etc), who will can solve it properly.

Process

First, we are going to create an Azure Notebooks project where we can set up a Notebook server. Then we will create and upload some files based on Python to deploy a Text Summarization service in an Azure Machine Learning Workspace. After that, we will construct and train a simple deep neural network classification model, with Keras and Tensorflow, that will classify the hotel customers reviews.

Also, we will perform the integration with the Azure Cognitive Services along with the Azure Machine Learning. We will use Computer Vision API to collect information from pictures or photographs and Text Analytics API to extract data from human utterances.

Advantages

The principal advantage of using Cognitive Services and Deep Learning in this case is the possibility of identifying very quickly the reviews or comments that needs to be carefully manage by a human been because of its importance or its urgency.

Even if tons and tons of messages arrive, we can be sure that the most critical messages will be identified and treated immediately and the rest of them will be classified and stored correctly.

This way a company will save money and resources and will be very much efficient attending customers’ requests.

Architecture

We tried to simulate hotel customer inputs from different sources (01) like Social Media, this hotel reviews (02) will be send to an Artificial Intelligent for its analysis.

We used Azure Notebooks with Jupyter to create, train and deploy the services we need. Ones of them are Containerized Services (03) we trained with Deep Learning like the Summarize Service (05) and the Classification Service (06). Others are Cognitive Services (04) we used directly from Microsoft like Computer Vision (07) and Text Analytics (08).

If you want to read more go to this post.

Step by Step

1. How to create an Azure Notebooks project

First, we can access this Azure Notebooks Service by visiting https://notebooks.azure.com/ and then, we can create a new Azure Notebooks Project from “My Projects”.

If you want to see the whole process just go to this post.

2. How to construct a Deep Neural Network

Keras is a Python library that allows us to construct Deep Learning models. First, we must import all needed modules and download the text analytics files from our GitHub repository. The cells of the “.ipynb” script look like this:

import re
import nltk
import uuid
import os
import numpy as np
import pandas as pd
import tensorflow as tf
import keras
from keras import models, layers, optimizers, regularizers
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.utils import to_categorical

import urllib.request
data_location = './data'
base_data_url = 'https://raw.githubusercontent.com/idiWork/Experiment_102/master/resources/'
filesToDownload = ['reviews_labels.txt', 'reviews_texts.txt', 'contractions.py', 'textanalytics.py']

os.makedirs(data_location, exist_ok=True)

for file in filesToDownload:
    data_url = os.path.join(base_data_url, file)
    local_file_path = os.path.join(data_location, file)
    urllib.request.urlretrieve(data_url, local_file_path)

nltk.download('stopwords') 
nltk.download('punkt')

import sys
sys.path.append(data_location)
import textanalytics as ta

If you want to see the whole process just go to this post.

3. How to deploy Azure Cognitive Services

Computer Vision is an Azure Cognitive Service that allow us to get information from an image. It analyzes the image environment, detects objects inside them and classifies all of them by assigning tags and labels. We can find this resource in our Azure Portal under the “AI + Machine Learning” group:

After deploying the service, we must look and write down at the subscription key and the endpoint to integrate the service in our project.

If you want to see the whole process just go to this post.

4. How to use Microsoft Flow to send alerts

We have created a very simple C# console project to simulate the reception of hotel reviews. The program makes HTTP requests to the different services we have previously deployed to collect all the information from the image and text of the review. These services include containerized services like Summarize or Classification and Cognitive Services like Computer Vision or Text Analytics.

All review data will be stored in a SQL database. This is the database definition:

CREATE TABLE [dbo].[review](
    [id] [int] IDENTITY(1,1) NOT NULL,
    [text] [nvarchar](MAX) NULL,
    [class] [nvarchar](64) NULL,
    [sentiment] [nvarchar](64) NULL,
    [summary] [nvarchar](2048) NULL,
    [keys] [nvarchar](1024) NULL,
    [image] [nvarchar](1024) NULL,
    [captions] [nvarchar](1024) NULL,
    [categories] [nvarchar](1024) NULL,
    [tags] [nvarchar](1024) NULL,
    CONSTRAINT [PK_review] PRIMARY KEY ([id])

If you want to see the whole process just go to this post.

To keep reading about these topics, check out the following links:

Links

Azure Logic Apps

Using Blockchain & IoT during organ transportation

idiWork — Tue, 24 Sep 2019 07:46:36 +0000

Technology has made possible to imagine incredible things, but we cannot lose sight of the reason we create: people.

We must continue creating to improve our present and also our future, and one of the most important issues we can improve on is in the medical field. In idiWork we have imagined what it would be like to combine IoT with blockchain for the transport of volatile goods, such as organs in transplant surgeries.

Through IoT, we can have continuous information on the state of an organ. With Azure technology, we can transmit information about the organ in real time, and with blockchain, that it is transmitted under any circumstance and in a secure way.

In addition, through the smart contract that we will create with blockchain, we will maintain an meticulous record of each action that is performed throughout this process, in addition making sure that each party involved is synchronized, taking action at the time that is required.

This is the structure of the project, in which we can see how we obtain the information from the physical device, registering it properly and registering it...

... so that each event and useful information reaches all the users involved:

Do you want to know how we have achieved it? In idiWork you will find the step by step process in great detail, from the IoT device setting to the Azure functions development, to learn how this technology works and to continue creating.

Want to know more?

First part: IoT device setup 🔌

In the first part, we will make the IoT device setup through Azure IoT hub.

We register the device we work with, configuring its Wi-Fi connection and making possible the transmission of information with Azure services. You can even follow the process with the device in our video, an MxChip AZ3166. Check it out:

Second part: Blockchain Worbench App 🔒

In the second part, you will learn how to create a Blockchain Worbench App. Here we will detail one of the most important parts of this process: the smart contract. We can define all possible states of the organ that is transported, the minimum and maximum temperature, and the parts that operate in each state as well. Thus everything is coordinated to avoid errors, and facilitate all parties to act in the moment and in the way they should.

Third part: Azure Function App 💻

In the end, the third part will show how to create a Azure Function App to register all telemetries. We will specify the values of the telemetries that we register, and you can explore in detail the code in C#, where we see how to authorize the actions of the process and how the information is serialized or deserialized in JSON.

You can even see this experiment in action, don't miss it!

This is how we can imagine, build and create a great future.

Don't hesitate to visit the repository on GitHub, where we publish all our experiments:

idiWork / Experiment_101

Using Blockchain & Internet of Things During Organ Transportation

Experiment #101 - Organ Transportation

Using Blockchain & IoT During Organ Transportation.

Project Date:

May - August 2019

Services:

Azure IoT Central
Azure IoT Hub
Azure Event Hubs
Azure Functions
Azure Logic Apps
Azure Blockchain Workbench

Technology used:

Internet of Things
Blockchain

About

This Experiment researches the possibilities of combining Internet of Things (IoT) and Blockchain for transportation of volatile goods. IoT allows us to communicate with devices and collect ambient measurements like temperature and humidity from different sensors. The different components involved send and receive data and the Blockchain structure stores, transmits and verifies this data with integrity and in a secure way.

Idea

The experiment attempts to simulate the real time monitoring of environmental conditions in the transportation of volatile goods from one point to another, verifying at all times that the obtained values falls within the established security parameters.

Utility

For this experiment we chose the case…

View on GitHub

Thanks for reading, see you in the next experiment!

DEV Community: idiWork

Brand presence

About

Idea

Process

Utility

Introduction

Architecture

Preparation

Services

Analysis of the information

Understanding the information

Analysis

Global metrics – tab

Social media – tab

Twitch – tab

“TV Channel” tab

Links

Analysis of hot spots in physical stores

About

Idea

Process

Utility

Introduction

Description of the system

Architecture

Organization of each shop

Services

Facial Recognition

Computer Vision

Assembling the system

Analysis of the information

Sentiment and pattern analysis

Benefits

Other cases of use

Sports

Medicine

Farming

Transportation

More:

Links

Facial Recognition Access Control

About

Idea

Utility

Process

Advantages

Architecture

Step by Step

1. How to work with Face Cognitive Service

2. How to set up the IoT device hardware

3. How to pack and launch the IoT project

Links

Create a hotel customer reviews classification with Azure

About

Idea

Utility

Process

Advantages

Architecture

Step by Step

1. How to create an Azure Notebooks project

2. How to construct a Deep Neural Network

3. How to deploy Azure Cognitive Services

4. How to use Microsoft Flow to send alerts

Links

Using Blockchain & IoT during organ transportation

First part: IoT device setup 🔌

Second part: Blockchain Worbench App 🔒

Third part: Azure Function App 💻

This is how we can imagine, build and create a great future.

idiWork / Experiment_101

Using Blockchain & Internet of Things During Organ Transportation

Experiment #101 - Organ Transportation

Using Blockchain & IoT During Organ Transportation.

Project Date:

Services:

Technology used:

About

Idea