This is a submission for the Twilio Challenge
What I Built
The motivation behind this project stems from a deep desire to democratize the interview process and make it more accessible, personalized, and fair. Job interviews can be incredibly stressful, and traditional methods often fail to account for individual differences in language proficiency, communication style, and comfort levels. By harnessing the power of AI and advanced communication technologies, we can create an interview experience that not only assesses a candidate's qualifications but also provides real-time support and feedback. This platform aims to empower job seekers, helping them present their best selves and gain valuable insights to improve their performance. In a world where talent is everywhere but opportunities are not, this project seeks to bridge that gap and bring us closer to a more equitable job market.
So with this motivation, I have created an innovative AI-powered platform that personalizes the interview experience for job candidates. Users can fill out a survey detailing their job position, experience, skills, goals, and preferred language for the interview. Based on their responses, they can choose to conduct the interview via web browser or WhatsApp Business.
If a user opts for WhatsApp, they will receive interview questions generated by the Gemini API directly on WhatsApp, with the ability to respond and receive follow-up questions in their chosen language. Upon completion, they receive a voice call and a WhatsApp message with AI-generated feedback.
For those who prefer a web browser interview, the platform provides a video interface that opens the user's camera and microphone. Questions appear onscreen, and the AI analyzes body language using TensorFlow and PoseNet. Users receive real-time feedback on their posture and eye contact, can replay their responses, and view live transcripts of their speech. After the interview, they exit the video room and receive detailed feedback.
Demo
Multi Language Support-> Spanish
Source Code
Next-Gen AI Interview: Multilingual and Real-Time Analysis
This repository contains my submission for the Twilio Challenge.
What is this project about?
The motivation behind this project is to democratize the interview process, making it more accessible, personalized, and fair. Traditional interviews often fail to accommodate individual differences in language proficiency, communication styles, and comfort levels. By leveraging AI and advanced communication technologies, this platform aims to provide an interview experience that not only assesses a candidate's qualifications but also offers real-time support and feedback.
Key Features
- Personalized Interview Setup: Users fill out a survey detailing their job position, experience, skills, goals, and preferred language for the interview.
- Multilingual Support: Interviews can be conducted in the user's preferred language via WhatsApp Business or a web browser.
- AI-Generated Questions and Feedback: Gemini API generates context-specific interview questions and AI-generated feedback.
- Real-Time Body Language Analysis: TensorFlow and PoseNet analyze user posture and eye…
Twilio and AI
Twilio APIs form the backbone of my project, enabling seamless communication across different channels. Twilio WhatsApp Business API ensures smooth delivery of interview questions and responses, while Twilio Voice API handles voice feedback calls. Twilio Video API powers the browser-based video interviews, integrating with TensorFlow and PoseNet for real-time body language analysis. The Gemini API generates intelligent, context-specific questions and feedback, while Microsoft Text Translator ensures language accessibility. Together, these technologies create a cohesive and intuitive interview experience that is accessible, multilingual, and highly interactive.
Features
- Personalized Interview Setup
- Multilingual Support
- Dual Interview Modes (WhatsApp Business Mode, Web Browser Mode)
- AI-Generated Questions and Feedback
- Real-Time Body Language Analysis
- Replay and Transcript Features
- Users can replay their spoken answers to review performance.
- Live transcription of spoken responses displayed on screen.
- Seamless Integration with Twilio APIs
- Comprehensive Feedback
Tech Stack
- Frontend: React JS
- Backend: Express JS
- Styling: Tailwind CSS
- AI and ML: TensorFlow, PoseNet
- APIs:
-
Twilio APIs:
- WhatsApp Business API
- Voice API
- Video API
- Gemini API (for intelligent question generation and feedback)
- Microsoft Text Translator API (for multilingual support)
-
Twilio APIs:
Additional Prize Categories
Twilio Times Two: The project uses Twilio Whatsapp Business API, Twilio Voice API and Twilio Video API.
Impactful Innovators: This project is beneficial for job seekers that can use AI-driven, multilingual support and real-time feedback, empowers them to shine and bridging the gap between global talent and opportunities.
Entertaining Endeavors: The platform enhances the interview experience by incorporating interactive features like real-time body language analysis and replayable responses, making the process both informative and engaging for candidates.
<!-- Thanks for participating! →
Top comments (4)
It's insane how much you've been able to achieve. I have seen many interview-bots in the market and I have to say that this is quite near the top of the cream as far as interview-bots go.
Thank you so much @onesoltechnologies for your kind words! I'm thrilled to hear that you find my project impressive. It’s been a rewarding challenge to integrate these advanced technologies and create a platform that truly enhances the interview experience. Your feedback means a lot and motivates me to keep improving!
If a company cannot be bothered to devote time to interviewing candidates in person, it is not a company I want to work for. Being interviewed by AI is dehumanising and insulting.
@jonrandy I appreciate your perspective. Our platform is designed as a mock interview setup to help candidates build confidence and master their tone and communication skills. It's not meant to replace human interaction but to provide a supportive environment for practice and improvement before the actual in-person interview.