DEV Community

Suneet Patil
Suneet Patil

Posted on

Building AI driven Multi-modal Feedback Systems using Azure Bot Services and RPA


Currently if you want to provide a feedback for items ordered, it’s a big tedious process of logging into apps/websites and filling the lengthy forms and submit or calling the customer care and following the IVR and then submitting the feedback.

With the addiction to internet, customers expect to contact the organizations very easily and quick response to get acknowledgement for the feedback. Many a times due to traditional way of handling the feedback would take time and end up failing to act of feedback and lose the customer base

Proposed Solution:

There is a need for AI driven Feedback system which eases the feedback submission process and successive updates on the actions taken to customer will not only satisfy the Customer but also encourages for repetitive business.

Organizations are turning to artificial intelligence (AI) solutions that make use of AI agents, commonly known as bots to provide a first-line of automated support through the full range of channels that we use to communicate. Customers can converse with the Bots in natural language.

  1. Customers can scan the QR Code to launch the feedback Bot from their Mobile (opens as part of Mobile's web browser)
  2. Take a snap of their bill and uploads to Bot
  3. Bot greets the User with Name by extracting the bill Number and fetching from back-end servers. Also, displays list to items(ordered as part of bill) as buttons to choose to give a feedback
  4. Customer chooses the item on which they wants to provide feedback
  5. Bot requests for feedback
  6. Customer can provide feedback by typing or they can press on mic and provide the voice input
  7. Bot Analysis the Customer's feedback by sentiment analyses and decides whether it is feedback or the issue. If Issue, raise it with the back-end system and intimate the customer by sending mail to registered email(taken from bill id).
  8. Acknowledge the Customer once the issue is resolved by email keeping them updated on the necessary actions taken for the feedback provided

This way we can reduce the traditional tedious process of Customer providing feedback in just 4 simple steps and updates on feedback status.



Technical Implementation:

List of Services used for the implementation,

  • Azure Bot Service. This service provides a framework for developing, publishing, and managing bots on Azure. Create a custom bot using Microsoft Bot Framework SDK to build a custom conversation to greet user, collects bill, integrates with Azure Custom Vision API to read the data, Collects feedback, Connects with Sentiment analysis, 3rd party ticketing systems like ServiceNow and Gmail services to send email
  • Azure Cognitive Services: This service determines the language of the text, perform sentiment analysis on text to determine a positive or negative sentiment, extracts key phrases in the text. Used this service to send the customer feedback text to Azure cognitive service to segregate the feedbacks with the issues reported. If its positive or neutral sentiment, acknowledge the Customer for the feedback. If negative sentiment, raise a ticket in one of ticketing platforms as issue and acknowledge the Customer with ticketing details to their email with necessary steps taken to rectify and correct it.
  • Azure Custom Vision Services: This service extract information from images; a common task in many artificial intelligence (AI) scenarios. Used this service to send the customer uploaded bill to exact the bill number and other details and pass to backend systems to fetch the customer details like Name, Email and order details for further use.
  • Gmail: This is the free email service developed by Google Used this service to send email updates to Customer about the issue details raised by them and action taken to rectify it.
  • ServiceNow: a ticketing tool used to manage Technology Service Management tasks. Used this service to raise the Customer issues and create a workflows to take necessary actions.


QuickFed : AI driven automated feedback system using Azure bot services and Automation Anywhere

Top comments (0)