Helpline Triager: Your buddy during distress

This is a submission for Twilio Challenge v24.06.12

What I Built

Helpline Triager
While reporting or seeking help in a distress situations, finding the appropriate help can be challenging. This project aims to streamline the process by providing a single helpline number that redirects caller-provided information to the relevant helpline. Such an approach address common obstacles faced during distress situations such as lack of internet access, reluctance to self-identify, and the precious time-consuming task of locating the correct helpline number.
By offering a centralized and user-friendly solution, this project aims to facilitate prompt access to the necessary support during times of distress.

Technologies Used
Backend: Flask, gevent
External APIs:

• Twilio Api for messaging, call
• OpenAI Api for AI responses, Whisper for transcription
• Google text-to-speech for text to speech

Demo

As this is a backend app only, no front-end demo is present. One can easily integrate this as a service.

Attaching one of the messages that received using this backend app.

Twilio and AI

I leveraged Twilio's powerful communication APIs(both call and messaging API), to handle incoming calling calls in the backend app. The use of AI in the app is mostly two-fold

1. Caller agent: An AI agent is used as both receiver (while getting distress call from end-user) and caller/message-sender (while directly appropriate helpline).
2. Agent Prompts: Different agent prompts are used to - (a) identify distress type, (b) extract important information, and (c) summarize the distress call.

Additional Prize Categories

This submission qualifies for the following additional prize categories:

Twilio Times Two: The project utilizes both Twilio's calling and messaging APIs to enhance our Notion extension.

Impactful Innovators: This project can make up the split-second advantage required to get the essential support during critical moments of distress such as fire, health emergency, theft, etc.

Github Repo

thepurpleowl / Helpline_Triager

Use ChatGPT over Twilio to create an AI phone agent to redirect distress calls to relevant helpline.

Helpline Triager

Use ChatGPT over Twilio to create an AI phone agent to redirect distress calls to relevant helpline.

While reporting or seeking help in a distress situations, finding the appropriate help can be challenging. This project aims to streamline the process by providing a single helpline number that redirects caller-provided information to the relevant helpline Such an approach address common obstacles faced during distress situations such as lack of internet access, reluctance to self-identify, and the precious time-consuming task of locating the correct helpline number By offering a centralized and user-friendly solution, this project aims to facilitate prompt access to the necessary support during times of distress.

How it works

Twilio Webhook -> Flask app -> Twilio Media Stream (websocket) -> Whisper -> ChatGPT API -> Google TTS -> Twilio Play Audio

How to use

1. pip install -r requirements.txt
2. Set environment variables in keys.env file

OPENAI_API_KEY=
TWILIO_ACCOUNT_SID=
TWILIO_AUTH_TOKEN=
TWILIO_PHONE_NUMBER=+1...

…

View on GitHub