AI Podcasts powered by Project X 🎼

This is a submission for Twilio Challenge v24.06.12

What Was Built

Project X is an innovative learning platform designed to revolutionize the educational experience by leveraging advanced technologies like AI and cloud computing. Users are empowered to effortlessly gather relevant information from various sources, creating an interactive and educational module. The AI Podcasts feature, powered by Project X, allows for the specification of a topic and character personas to generate engaging podcasts. Twilio APIs are used to enable interactive discussions, enhancing the learning experience.

Demo

Link to the app: https://projectx.nishithp.dev

NishithP2004 / AI_Podcasts

NishithP2004 / projectx-frontend-dev

Project X

The power of knowledge in the palm of your hands...

Execution Steps

1. Clone Frontend Repository:

   • Clone the frontend repository from GitHub using the following command

     git clone https://github.com/NishithP2004/projectx-frontend.git

2. Clone Backend Repository:

   • Similarly, clone the backend repository from GitHub:

     git clone https://github.com/NishithP2004/projectx-backend.git

3. Navigate to Frontend Directory:

   • Move into the projectx-frontend directory:

     cd projectx-frontend

4. Create Environment Variables File:

   • Create a .env file in the root of the projectx-frontend directory.
   • Copy the contents of the .env.example file provided below and paste it into the .env file.
   • Fill in the required values for your environment variables.

5. Start Frontend Service:

   • Run Docker Compose to start the frontend service:

     docker-compose up

6. Navigate to Backend Directory:

   • Switch to the projectx-backend directory:

     cd ../projectx-backend

7. Start Backend Service:

   • Ensure you have Microsoft Azure CLI tools installed.
   • Start the backend service using the Azure Functions Core Tools:

     func start

---

.env.example (Common to both frontend & backend)

G_SEARCH_KEY=
G_CX=

…

View on GitHub

NishithP2004 / projectx-backend-dev

projectx-backend

Execution Steps

1. Clone Backend Repository:

   • Clone the backend repository from GitHub using the following command

     git clone https://github.com/NishithP2004/projectx-backend.git

2. Navigate to Backend Directory:

   • Move into the projectx-backend directory:

     cd projectx-backend

3. Create Environment Variables File:

   • Create a .env file in the root of the projectx-backend directory.
   • Copy the contents of the .env.example file provided below and paste it into the .env file.
   • Fill in the required values for your environment variables. Refer to the comments in the .env.example file for guidance on each variable.

4. Install Dependencies:

   • Install the necessary dependencies by running:

     npm install

5. Start Backend Service:

   • Ensure you have Microsoft Azure CLI tools installed.
   • Start the backend service using the Azure Functions Core Tools:

     func start

6. Access Backend Endpoints:

   • Once the backend service is running, you can access the various API endpoints provided by the backend component.

---

.env.example (Common to both frontend & backend)

G_SEARCH_KEY=
G_CX=
BING_CONFIG_ID=
BING_API_KEY=
GOOGLE_PALM_API_KEY=
GOOGLE_VERTEX_AI_TOKEN=

…

View on GitHub

Features Offered by Project X (An Overview)

Twilio and AI

In Project X, Twilio's Programmable Voice and Messages API (WhatsApp Sandbox) were utilized to implement the Interactive AI Discussions feature. This allows for the scheduling of interactive discussions, changing the direction of the discussion, and receiving a summary and transcript via WhatsApp. The AI element is powered by Azure's OpenAI (GPT-4) for generating content and Azure Speech Services for text-to-speech conversion, making the podcasts natural and engaging.

How It Works:

1. Podcast Initiation: Users specify a topic and character personas for the podcast.
2. Podcast Generation: Using Azure OpenAI (GPT-4), the script for the podcast is generated, grounded in the data provided by the user as part of the courses created on the platform.
3. Interactive Discussion: Interaction with the podcast is enabled through Twilio APIs (barge), altering the discussion direction and resulting in a dynamic learning experience.
4. Summary and Transcript: At the end of the discussion, a summary and transcript of the session are sent via WhatsApp. The same will be made available in the online platform as a normal podcast for future use.

Additional Prize Categories

Twilio Times Two

Two Twilio APIs are utilized in our submission:

1. Twilio Programmable Voice: For interactive podcast discussions.
2. Twilio Messages API (WhatsApp Sandbox): For delivering summaries and transcripts to users.

Impactful Innovators

A significant educational gap is addressed by Project X by providing dynamic, interactive learning resources that go beyond traditional static references. This positive impact on education can enhance the learning process, making it more engaging and effective for students.

Entertaining Endeavors

A creative and entertaining way to learn is introduced by the AI Podcasts feature. By allowing the specification of character personas, including fictional or real-world entities, the platform makes learning fun and engaging. The interactive discussions add an element of unpredictability and excitement to the learning process.

Imagine quantum physics is the topic of interest, but instead of a dry lecture, a podcast featuring Albert Einstein and Sheldon Cooper from "The Big Bang Theory" is chosen. As the podcast starts, Einstein provides deep insights into quantum theory, while Sheldon frequently interrupts with quirky, humorous comments, asking, "Why does everything have to be so... theoretical? Can't we just solve it with string cheese and a whiteboard?"

Halfway through the podcast, the direction of the discussion is changed using Twilio’s interactive feature. The conversation is steered toward the practical applications of quantum physics in everyday life. Sheldon, being Sheldon, starts brainstorming how quantum entanglement could be used to improve Wi-Fi speeds in his apartment. The podcast concludes with Einstein trying to keep a straight face while explaining the serious implications of Sheldon's idea.

At the end of this hilarious and informative session, a summary and transcript are received via WhatsApp, complete with Einstein’s theoretical explanations and Sheldon’s comical commentary. This not only makes learning quantum physics enjoyable but also leaves a memorable and engaging learning experience.

Conclusion

The AI Podcasts feature of Project X, powered by Twilio and Azure AI technologies, transforms the traditional learning experience into an interactive, engaging, and impactful journey. By leveraging Twilio's capabilities, a platform that not only educates but also entertains has been created, pushing the boundaries of creativity in the educational landscape.

Disclaimer: This project utilizes a free trial account along with the Twilio WhatsApp Sandbox API & Twilio Voice API, which requires a verified phone number for service access.