About
There are countless processes in a physical store that produce data that can be used and exploited to generate or increase sales processes.
In addition to measuring the time spent in the store and register whether the visitor ends up buying something or not, the purpose of experiment #205 is to use facial recognition services to locate hot spots in the store, sentiment analysis to analyze the level of satisfaction of the client, and Cognitive Services to analyze bags or objects from other brands.
With all this information it’s possible to calculate a predictive stock to automatically supply the store.
Idea
The idea arises from the need and the lack of use of the information that can be exploited in physical stores. Nowadays data-driven decision making is crucial for the proper development of a business and thanks to AI this reality can be possible.
Process
First, we will use the store’s cameras to analyze the images. Then, we will develop a software that collects all the information and sends it to the cloud to be analyzed in the Cognitive Services. Once in the cloud, all the necessary analysis, the entire ETL process and a further integration with the corporate DataWarehouse will be carried out and utilized through PowerBI.
Utility
Find correlations between the average time spent in a store and the probability that a purchase happens, detect heat sources and patterns for possible rearrangements of the layout, predict stocks, analyse customer sentiment to identify possible dissatisfaction and detect brands that customers consume for segmentation.
If you want to read more go to this post.
Introduction
For many people, artificial intelligence is something reserved. And for few, a future that is yet to come. But the reality is different, since we can see around us a huge number of examples where artificial intelligence is part of our daily lives. Who has not seen a series in Netflix or booked a hotel room in Booking suggested by the product recommender? Or even used Google to search the internet.
On the other hand, as subject matter experts, we are used to companies and large organizations that do not take full advantage of their data, since data is, without a doubt, ‘the new oil’. Extracting data and making the most out of it is a complex task but, if done in the right way, the return on investment is more than guaranteed.
Today we will focus on unraveling a system capable of tracing the patterns that people follow at a shopping center; analyzing their feelings at the arrival and departure, whether they go to the fitting room, buy something and much more. This procedure is carried out with Azure and the results are shown in Power BI.
Description of the system
Data from the shopping center`s security cameras are rarely exploited. Thanks to advances in technology, we can study our consumers, not only when they interact through our e-commerce platforms but also in the shops.
The system will answer the following questions:
- How many people have entered the shop?
- How long have they stayed inside?
- Which departments has the person visited?
- How much time has the person spent in each department?
- Has the person been in the fitting room?
- Has the person bought anything?
- How long has the person been queuing to pay?
- What is the person’s feeling (positive/negative, maybe surprise at a price, etc.) throughout the visit to the shop?
- Which parts of the store are most often visited?
- Which bags does the person have?
Architecture
The simplified architecture to meet all the needs and answer the previous questions would be this one:
We would start from the base of having n number of shops. Each shop would have several security cameras that will be used to trace the patterns and routes of each person.
At the shop’s closing time (although it could also be done in real time), all the data stored by the security cameras would be sent to Azure’s services for further analysis.
Let’s see it in detail:
Each shop will have a database where the different data generated throughout the day will be stored. At the end of the day, an automatic process will send the images captured by the cameras to be analyzed in Azure. In order to meet the business needs, the Face and Computer Vision tools will be used to answer all the target questions that were raised before.
Finally, all the data generated by Azure Cognitive Services will be stored in an Azure SQL Server in PaaS, for later analysis in Power BI and, thus, making it available for the analysis department.
Organization of each shop
Each store will need to be divided into n number of areas so that we can draw a later analysis focused on the desired direction.
For example, in a shop like this:
The cameras would need to be strategically placed in order to visualize the shop’s:
- Aisles
- Entry and exit of the shop
- Entry and exit of the fitting rooms
- Cash desks
In addition, the shop needs to be also divided into different areas, for example, in red, green and blue zones, that may correspond to men’s, women’s and children’s departments. Like this:
Services
As discussed in the above, we will use Microsoft Cognitive Services for image processing:
Within the wide spectrum it offers us, we will use the Facial Recognition and Computer Vision services.
Facial Recognition
The Facial Recognition service detects the faces of the people who pass through the space we are studying. In addition to identifying them and being able to differentiate them from each other, it also recognizes the facial expressions and provides us the emotion the person is feeling at that specific moment.
Computer Vision
The Computer Vision service identifies the brands of the bags that our customers carry with them:
In addition, the Computer Vision service locates the people in our area, giving us the x and y coordinates of the establishment in order to identify the patterns, hot spots and much more.
Assembling the system
Now that we know what the final goal is and the parts that compose it, we’re going to try to assemble the whole system so that it’s able to function as a whole and can provide us the information we need to analyze our business so we can gain a competitive advantage:
- As discussed earlier, the first step is to have each of the stores divided into sections that we are interested in studying. In addition to this, we would conduct a study to identify how many cameras would be needed and where they should be located to be able to control the entire space.
- Once this is done, the next step would be to have a system in charge of collecting the photos of each of the cameras along with information like the time, camera from which it was taken, and other metadata that interests us. All the information will be stored in a database (for example, in an instance of SQL Server) that will be available in each store.
- At the end of each day, there must be a process that will take all the information generated on that day and send it to our services in Azure so we can identify faces, emotions, brands, location in the store, etc.
- Finally, all the information generated by the services will be stored in a centralized database (on an Azure SQL Server instance in PaaS) in order to perform the relevant analyses.
With the entry of the new GDPR and to ensure that we don’t send images over the network with customers who haven’t given their explicit consent, we may slightly change the previous assembly:
- The first step is the same. We will have each store divided into sections and we would conduct a study to identify how many cameras would be needed and where they should be located.
- We would need exactly the same system to collect the photos of each of the cameras along with the information about the time and camera from which it was taken and other metadata that interests us. All this information will be stored in the database that will be available in each store. This point would remain intact in both methods.
- At the end of each day, there will be a process that will take all the information generated on that day and process it on the same store system, so we can bring the models of the three Azure services mentioned earlier to process the images as an IoT system in the Edge. With this technique the faces will be stored as a unique, irreversible, and anonymous identifier (for example, as a hash).
- Once we have processed the images, the information will be stored in the same database where the images and raw data have been stored.
- At this point we could delete all raw information so that we would only have anonymous data. What’s more, we could make a third approach that would be to process the images in real time, not having to store any raw images and eliminate any possibility of having any privacy-related problems.
- Finally, once we have stored all the processing information in the database, we will run one last process before the analysis which is an ETL (Extract, Transform and Load) of Business Intelligence to gather all the processed data from all the stores into a centralized database in Azure (in an Azure SQL Server).
Analysis of the information
Once we have assembled the architecture and processed the data using the Cognitive Services,the next step is crucial; we must extract information of that preprocessed data in order to bring it to the knowledge of analysts, who will use it for decision-making.
To address this last phase, we’ve decided to make use of a great ally like PowerBI. The report is as follows:
We have numbered it in such a way that it’s easier to explain and locate the items in the report.
- A bar chart in which we see the percentage of the brands of the bags that the customers carried. In this case almost 80% were Zara's.
- We can set different target values. In this case, we have established as our target that 50% of people who enter the store should buy an item. In this example 51.61% of the people ended up buying something.
- In addition to the sales objectives, we have also observed that there may be a correlation between the purchase and the visit to the fitting room. That’s why we established as a target that at least 40% of customers should go through the fitting room. In our case, 36.11% have done so; we’re a little below the target.
- We are also interested in having positive feelings within our store. One of the moments where most discontentment occurs is while waiting. So to do this, we have set a limit of 10 minutes of queue. On average, our customers spent 12.24 minutes queuing, a value somewhat above our goal.
- On the other hand, another aspect that also has a correlation with the final purchase is the time spent in the store. We have set 15 minutes as the minimum time a person should spend in the store. In this case, the average time spent in the store was 20.67 minutes, so this target was achieved.
- We also have some gender statistics. In our store 60% of the visitors are women and 40% are men.
- As for age, we can see how men and women are, on average, 34 and 44 years old respectively.
- As for the areas we defined earlier, we can see the percentage of people who passed through the different areas to take a look at the products. In the case of zone 1, 78% of the people who entered the store ended up passing through zone 1.
- In the case of zone 2, 60%
- Finally, in the case of zone 3, only 28% ended up visiting the zone. This may indicate that the area is not well designed, signalized or just doesn’t have very attractive products.
- In this bar chart we are interested in seeing the total number of people who entered the store each day, and the predominated feeling during the day. This can help us detect potential key days when more staff or better organization is needed. In addition to this, also observe peaks (both high and low) of visits.
- Finally, a filter to segment by date.
In point 11, we see that there is a day with a predominantly negative feeling. We’re talking about January 31:
When we filter it we can draw numerous conclusions:
- As we have said before, the first thing is that the prevailing feeling on this day is negative.
- We can see how the percentage of purchases is 68.3%, well above average (51.61%)
- The visits to the fitting rooms have reduced to 31.3%. This can happen because we are having a special Sales day, where it comes out more profitable for customers to buy and then return than waiting in line to get to the fitting room.
- Finally, the average time spent queuing has increased dramatically to 42.3 minutes
Sentiment and pattern analysis
An example of a successful purchase, without any negative emotions or highlights:
- In this chart we can see the path that the person does within our establishment. We can also see where the person is and how much time he/she spends in the same place. This chart also shows the emotions the visitor is having throughout its stay. We see how in one of the points it reflects surprise; probably because the price of something he/she likes is lower than his/her expectation. Also, after leaving the fitting room the visitor seems happy with the result.
- Slicer in charge of making the timeline move forward so that you can see chart 1 dynamically, making it possible to visualize the person’s journey throughout the store as if they did it for real.
- We can also see the actions this person has taken. In this case, the visitor made a purchase (the garment that surprised) and a return.
- The brands on the bags with which the visitor entered
- Gender data
- Age of the visitor
- The time spent in the store. In this case it has been higher than the minimum of 15 minutes that we set before
- The time spent in the queue, this being 8 minutes, less than our maximum.
- And finally, we have at our disposal a slicer for filters per person.
Next, let’s look at an example of a negative situation in the establishment:
This visitor, like the previous one, found a garment that was intriguing and, after passing through the fitting room, was convinced enough to buy it.
However, we can see how at the end she didn’t take any action. If we look closely at the pivot chart, just as she was queuing her feelings changed drastically first to anger and finally to disgust.
And we can clearly see how this person was in the establishment for 32 minutes, and 24 of them she passed waiting in line. A time period that is unrealistic to leave satisfied customers or create pleasant emotions in the establishment.
These types of reports are very close to those of traditional Business Intelligence, where we have graphs, metrics, KPIs,etc. to analyze our business. We can add as many as our analysis and business team needs to make the right decision. We have opted for these as an example, but there are countless studies and metrics to calculate the fruit of the idiosyncrasies of each establishment.
Benefits
- Different KPIs for sales and stock
- Sales Forecast
- Intelligent detection of areas of improvement
- Suggestions of promotions
- Seller analysis
- Customer behavior analysis
Other cases of use
We have adapted this experiment to the retail industry as we - think it’s a case in which it could be applied perfectly in order to get great results and a great return on investment.
However, there are numerous examples where this system could also be implemented:
Sports
Track athletes, their relative position and movements they make when scoring. Analyze correlations and build the key to success.
In addition, we could also analyze the audience that attend these events, their patterns, behavior, etc.
Medicine
We know that medicine is one of the sectors where AI is very present. This system could provide us with disease control, tracking of people with the disease and much more. This could help to prevent and/or combat diseases in a much more effective way.
Farming
Although it may seem like a very conservative sector, the use of new technologies is the tendency. The competitive advantages that can be achieved through its inclusion are considerable.
This system would allow us to study and control how animals move,what they eat, whether they have drunk enough water or not, and if they act accordingly.
Transportation
In the transport sector this system also can have a lot of use. In any space where there is movement of people, vehicles or objects it will allow us to analyze all the information completely autonomously and draw conclusions.
More:
Traceability of people in department stores, where there are a large number of stores
Traceability at airports to maximize their economic performance through passenger purchases
Covid-19 traceability
Optimization of activities (both lucrative and leisure) on cruise ships
Optimization and time waiting in line
Traceability in supermarkets
If you want to read more go to this post.
Top comments (0)