This is a submission for the AssemblyAI Challenge : Sophisticated Speech-to-Text.
What I Built
FluentAI is a cutting-edge AI-powered application designed to make English learning fun, engaging, and personalized. Whether you want to refine your reading, writing, or speaking skills, FluentAI offers tailored exercises, instant grading, and detailed feedback powered by advanced AI. It’s your one-stop solution to transform how you communicate in English!
How it works
Sign Up and Choose Your Focus
Start by signing up and creating your profile. Your details are securely stored in a PostgreSQL database for tracking your progress and storing feedback. Once logged in, select the area you want to work on: Reading, Writing, or Speaking.
Reading Practice
- Read aloud the given passage and record your audio.
- Once submitted, the audio file is processed:
- Transcribed into text using AssemblyAI Universal.
- Sent to Ollama Mistral Model for analysis.
- Receive scores and feedback on parameters like grammar, pronunciation, and tone of speech, all of which are updated in the database for future tracking.
Speaking Practice
- Choose a topic (or freestyle about your day!).
- Record and submit your audio.
- The same transcription and analysis process is applied to assess your fluency, clarity, and intonation.
- Personalized feedback and grades are stored in the database to help you chart your improvement.
Writing Practice
- Submit your written piece—be it an essay, story, or just a random thought.
- Your submission is sent to Ollama Mistral Model for a detailed evaluation based on parameters like structure, grammar, and coherence.
- Feedback is updated in the database to help you keep track of your writing progress.
PS: All your scores and feedback live safely in the database—no public displays of your grammar flubs or poetic masterpieces. 😅
Demo
checkout the live demo here:
Disclaimer: The responses may take around 5-10 min to generate as the server is deployed on AWS t2.large
(PS : I'm broke hehe..)
Landing Page
Reading section
Writing section
Speaking section
Feedback section
Source Code:
Ritesh2351235
/
Fluentai
FluentAI is an AI-driven platform to enhance English communication skills through personalized practice in reading, writing, and speaking. It provides detailed feedback and progress tracking to help users improve effectively.
FluentAI
FluentAI is an AI-powered application that helps users improve their English communication skills by practicing reading, writing, and speaking. The app provides personalized feedback and grades for each exercise, helping users identify areas of improvement and track progress over time.
Features
- Reading Practice: Read passages aloud, get feedback on grammar, pronunciation, and tone.
- Speaking Practice: Speak on a topic, receive scores on fluency, clarity, and intonation.
- Writing Practice: Submit written content and get detailed feedback on structure, grammar, and coherence.
Tech Stack
-
Frontend:
- React
- Tailwind CSS
- Shadcn Components
- GSAP (for animations)
-
Backend:
- Node.js
- Express
- AssemblyAI API (audio transcription)
- Ollama Mistral Model (AI evaluation and feedback)
-
Database & Hosting:
- PostgreSQL (Aiven) for user data and progress tracking
- AWS EC2 (t2.large) for backend deployment
- Vercel for frontend deployment
Feel free to contribute or report issues to help enhance FluentAI!
Journey
FluentAI was built to address a pressing challenge: providing an affordable, interactive way for users to improve their English communication skills. The integration of AssemblyAI’s Universal Speech-to-Text Model was crucial in bringing this vision to life.
AssemblyAI is the backbone of the audio transcription process, ensuring high accuracy and quick results.
The Ollama Mistral Model acts as the smart assistant, analyzing text and providing context-aware feedback.
It wasn’t just about building an app—it was about creating a seamless experience where users could learn, grow, and gain confidence in their abilities.
Tools Used
Building FluentAI involved a stack of powerful tools and technologies:
Frontend
- React: For crafting a dynamic and responsive user interface.
- Tailwind CSS: A utility-first framework to ensure sleek and modern designs.
- Shadcn: For reusable and aesthetic UI components.
- GSAP: For animations that make the app lively and engaging.
Backend
- Express + Node.js: For handling API requests and business logic.
- AssemblyAI: Converts audio into text with remarkable accuracy.
- Ollama Model (Mistral): Analyzes transcripts and text for personalized grading and feedback.
Infrastructure
- AWS EC2 (t2.large): For robust and scalable server hosting.(I'm Kinda broke, but this does the job..)
- Aiven Database: Stores user-profiles and progress securely.
- Vercel: Deploys the frontend for a fast and reliable user experience.
Final Thoughts
Building FluentAI has been an incredible journey, blending AI, creativity, and technology to address a real-world need. Whether you’re prepping for a presentation, working on essays, or simply polishing your conversational skills, FluentAI has your back.
With AI-driven grading and personalized feedback, learning English is no longer daunting—it’s a journey of constant growth and discovery.
Future Plans
I aim to make FluentAI even smarter with features like:
- Daily progress reports.
- AI-curated exercises based on your learning style.
- Gamified challenges to keep the learning fun and rewarding.
Top comments (2)
Hey Ritesh, this is something exciting and I look forward to explore it !!
Still amazed by how useful nutritionai is.
Good work!!