This is a submission for the AssemblyAI Voice Agents Challenge
๐ก What I Built
A real-time, AI-powered academic listening coach designed to help you:
- Grasp any concept
- Articulate it in your own words
- Get real-time feedback from an AI mentor trained to respond like a domain-specific educator
Imagine a personalized Listening Toastmasters for academics, one that:
โ
Listens while you speak
โ
Transcribes in real-time
โ
Analyzes your response
โ
Gives constructive feedback
โ
Grades your clarity, tone, and structure, not you ๐
Perfect for viva prep, thesis defense, placement interviews, just better grasping a concept or topic, or even explaining tough concepts out loud.
โจ Why I Built It
I've always craved a mentor who could truly adapt to me -
One who listens without judgment.
Who cares how I speak, not just what I say.
Who waits when I pause. And helps me find the words when I blank out.
As a student juggling placements, exams,hackathons and life, I often find myself:
- Mumbling under pressure
- Rambling mid-response
- Or going completely blank during interviews
So, I built this for that version of me.
The nervous student. The silent developer.
The person who knows the answer, but just canโt say it clearly.
This is more than a tool.
Itโs a gentle, nerdy best friend in your laptop, reminding you:
โYouโve got this. Just speak, Iโll help you shape it.โ
Oh, and if you're overusing filler words?
Itโll lovingly ๐ roast you:
โBestie, you just said โummโ 27 times. Letโs fix that.โ
๐ ๏ธ Features
1๏ธโฃ ๐๏ธ Mic On, Brain On
โLive voice input straight from your browser (no app install needed!)
2๏ธโฃ โ๏ธ Real-Time Whispering
โInstant speech-to-text via โก AssemblyAIโs Streaming API
3๏ธโฃ ๐ Instant Report Card
Get scored out of 10 on key communication metrics:
- ๐ฃ๏ธ Fluency
- ๐งฉ Coherence
- ๐ Redundancy
- ๐ง Technical Depth
- ๐ช Confidence Markers
๐ Delivered in real-time โ your growth, visualized.
4๏ธโฃ ๐ AI Educator Mode
โYour speech gets evaluated like you're explaining to a domain expert (Groq)
5๏ธโฃ ๐ Retry Until Itโs Right
โStumble? Speak again. Smarter each time. ๐
6๏ธโฃ ๐ฏ Focused Solo Practice
โA quiet dojo to train your mind-mouth connection ๐ค๐งโโ๏ธ
7๏ธโฃ ๐งช Built for the Serious Learners
โIdeal for:
โโ- ๐งฌ Viva / Thesis prep
โโ- ๐งโ๐ป Tech interviews
โโ- ๐ Academic presentations
โโ- ๐ค Fluency drills
โโ- โจ Better grasping any topic
8๏ธโฃ ๐ป Minimal UI, Max Results
โNo distractions. Just you, your thoughts, and your growth ๐ฅ
๐ฌ Demo
Here's my live project:- Grasp Articulate Refine
โ ๏ธ Works best in Chrome. Firefox sulks. Brave is brave. Safari is... shy.
You can check me showcasing my project here:-
GitHub Repository
๐๐
๐ง โจ๐ Grasp Articulate Refine
Your smart study coach, powered by AI - designed to help you truly understand what you learn, speak it with confidence, and get thoughtful feedback so you grow smarter, faster.
My project at a glimpse:-
Check it out here live:- Grasp Articulate Refine
โจ Features
- Adaptive Content Generation: Creates 2000-3000 word educational content tailored to your academic level
- Voice-Based Assessment: Uses Assembly AI for speech-to-text transcription
- AI-Powered Analysis: Acts as a globally renowned educator providing detailed feedback
- Intelligent Grading: Grades responses out of 10 with detailed explanations
- Progress Tracking: Students must score 9+ to advance to next topics
- Celebration System: 3-second emoji overlay for excellent performance (๐ฅณ๐๐)
- Mobile Responsive: Darker blue theme with high contrast design
- Real References: Provides working, relevant reference links for the explanation provided
- Multiple Academic Levels: High School, Undergraduate, Graduate, Professional
- Custom Subjectโฆ
You can check out my repo above if you are more of a code person, or want to analyse my code ๐ค, get inspiration, fork it, clone it, and work on it on your device locally.
Technical Implementation & AssemblyAI Integration
Here are the code snippets demonstrating the technical implementation and AssemblyAI integration in this project:-
๐ฏ 1. AssemblyAI Initialization & Configuration
python
# utils/voice_manager.py - AssemblyAI Setup
import assemblyai as aai
class VoiceManager:
def __init__(self, api_keys: Dict[str, str]):
self.api_keys = api_keys
self.assemblyai_available = False
self._init_assemblyai()
def _init_assemblyai(self):
if ASSEMBLYAI_AVAILABLE and self.api_keys.get('ASSEMBLYAI_API_KEY'):
try:
aai.settings.api_key = self.api_keys['ASSEMBLYAI_API_KEY']
test_config = aai.TranscriptionConfig(
language_detection=True,
punctuate=True,
format_text=True,
speaker_labels=False,
auto_highlights=False
)
self.assemblyai_available = True
print("โ
AssemblyAI initialized successfully")
except Exception as e:
print(f"โ AssemblyAI initialization failed: {e}")
self.assemblyai_available = False
Sets up AssemblyAI SDK with API key and configures transcription settings including language detection, punctuation, and text
formatting. Initializes the VoiceManager class with enhanced features optimized for educational content transcription.
๐ค 2. Core Audio Transcription Implementation
python
def transcribe_audio(self, audio_file_path: str) -> str:
if not os.path.exists(audio_file_path):
return "โ Audio file not found"
# Primary Method: AssemblyAI SDK
if self.assemblyai_available:
try:
print("๐ Trying AssemblyAI SDK...")
config = aai.TranscriptionConfig(
language_detection=True,
punctuate=True,
format_text=True,
speaker_labels=False,
auto_highlights=False )
transcriber = aai.Transcriber(config=config)
transcript = transcriber.transcribe(audio_file_path)
if transcript.status == "completed":
print("โ
AssemblyAI SDK transcription successful")
return self._clean_transcription(transcript.text)
elif transcript.status == "error":
print(f"โ AssemblyAI SDK error: {transcript.error}")
return f"โ Transcription error: {transcript.error}"
except Exception as e:
print(f"โ AssemblyAI SDK error: {e}")
# Fallback Method: Direct API
if self.api_keys.get('ASSEMBLYAI_API_KEY'):
try:
print("๐ Trying AssemblyAI Direct API...")
result = self._transcribe_with_api(audio_file_path)
if result and not result.startswith("โ"):
print("โ
AssemblyAI API transcription successful")
return self._clean_transcription(result)
except Exception as e:
print(f"โ AssemblyAI API error: {e}")
return "โ Transcription failed. Please check API configuration."
Main transcription function using dual-mode approach: primary AssemblyAI SDK method with fallback to direct API. Handles audio file
validation, processes transcription with enhanced configuration, and includes comprehensive error handling for reliable speech-to-
text conversion.
๐ง 3. Direct API Implementation with Enhanced Features
python
# utils/voice_manager.py - Direct API Implementation
def _transcribe_with_api(self, audio_file_path: str) -> str:
"""
Direct AssemblyAI API implementation with robust error handling
"""
try:
headers = {'authorization': self.api_keys['ASSEMBLYAI_API_KEY']}
print("๐ค Uploading audio file...")
with open(audio_file_path, 'rb') as f:
response = requests.post(
'https://api.assemblyai.com/v2/upload',
headers=headers,
files={'file': f},
timeout=60
)
if response.status_code != 200:
return f"โ Upload failed: {response.status_code} - {response.text}"
upload_url = response.json()['upload_url']
print(f"โ
File uploaded: {upload_url}")
print("๐ Requesting transcription...")
data = {
'audio_url': upload_url,
'language_detection': True,
'punctuate': True,
'format_text': True,
'speaker_labels': False,
'auto_highlights': False
}
response = requests.post(
'https://api.assemblyai.com/v2/transcript',
headers=headers,
json=data,
timeout=30
)
if response.status_code != 200:
return f"โ Transcription request failed: {response.status_code}"
transcript_id = response.json()['id']
print(f"๐ Transcription ID: {transcript_id}")
print("โณ Waiting for transcription to complete...")
max_attempts = 60 # 2-minute timeout
attempt = 0
while attempt < max_attempts:
response = requests.get(
f'https://api.assemblyai.com/v2/transcript/{transcript_id}',
headers=headers,
timeout=30
)
if response.status_code != 200:
return f"โ Status check failed: {response.status_code}"
result = response.json()
status = result['status']
if status == 'completed':
print("โ
Transcription completed")
return result['text'] or "โ No text in transcription result"
elif status == 'error':
error_msg = result.get('error', 'Unknown error')
return f"โ Transcription error: {error_msg}"
elif status in ['queued', 'processing']:
print(f"โณ Status: {status} (attempt {attempt + 1}/{max_attempts})")
import time
time.sleep(2) # 2-second polling interval
attempt += 1
else:
return f"โ Unknown status: {status}"
return "โ Transcription timeout - took too long to process"
except requests.exceptions.Timeout:
return "โ Request timeout - please try again"
except Exception as e:
return f"โ Unexpected error: {str(e)}"
Implements direct AssemblyAI API calls as fallback method. Handles file upload, transcription request with enhanced features, and
intelligent polling with 2-minute timeout. Provides robust error handling for network issues and API failures.
๐งน 4. Advanced Text Processing & Cleaning
python
# utils/voice_manager.py - Text Processing
def _clean_transcription(self, text: str) -> str:
if not text:
return "โ Empty transcription result"
text = text.strip()
text = re.sub(r'\s+', ' ', text)
text = re.sub(r'([.!?])\s*([a-z])',
lambda m: m.group(1) + ' ' + m.group(2).upper(), text)
if text and not text[0].isupper():
text = text[0].upper() + text[1:]
if text and text[-1] not in '.!?':
text += '.'
return text
def validate_audio_file(self, file_path: str) -> Dict[str, any]:
if not os.path.exists(file_path):
return {
'valid': False,
'error': 'File does not exist',
'file_size': 0
}
file_size = os.path.getsize(file_path)
max_size = 100 * 1024 * 1024 # 100MB limit
if file_size > max_size:
return {
'valid': False,
'error': f'File too large: {file_size / (1024*1024):.1f}MB (max 100MB)',
'file_size': file_size
}
if file_size < 1000: # Minimum 1KB
return {
'valid': False,
'error': 'File too small - may be empty or corrupted',
'file_size': file_size
}
return {
'valid': True,
'error': None,
'file_size': file_size,
'file_size_mb': file_size / (1024 * 1024)
}
Post-processes transcription results with text cleaning, whitespace normalization, sentence capitalization fixes, and proper
punctuation. Includes audio file validation checking size limits and file integrity for optimal transcription quality.
๐ 5. Flask Integration & API Endpoints
python
# app.py - Flask Integration
@app.route('/transcribe_audio', methods=['POST'])
def transcribe_audio():
try:
audio_file = request.files.get('audio')
if not audio_file:
return jsonify({'error': 'No audio file provided'}), 400
session_id = session.get('session_id', 'unknown')
temp_path = f"temp/audio_{session_id}.wav"
os.makedirs('temp', exist_ok=True)
audio_file.save(temp_path)
print(f"๐ Starting transcription of {temp_path}")
validation = voice_manager.validate_audio_file(temp_path)
if not validation['valid']:
if os.path.exists(temp_path):
os.remove(temp_path)
return jsonify({'error': validation['error']}), 400
transcription = voice_manager.transcribe_audio(temp_path)
if os.path.exists(temp_path):
os.remove(temp_path)
print(f"โ
Transcription result: {transcription[:100]}...")
return jsonify({
'success': True,
'transcription': transcription,
'file_size_mb': validation.get('file_size_mb', 0)
})
except Exception as e:
print(f"โ Transcription error: {e}")
return jsonify({'error': f'Transcription failed: {str(e)}'}), 500
@app.route('/voice_status')
def voice_status():
return jsonify(voice_manager.get_voice_status())
Flask endpoints for audio transcription with session-based temporary file handling. Includes comprehensive error handling, file
validation, and cleanup. Provides voice status endpoint for real-time feature availability monitoring and diagnostics.
๐ 6. Status Monitoring & Diagnostics
python
# utils/voice_manager.py - Status Monitoring
def get_voice_status(self) -> Dict[str, bool]:
return {
'assemblyai_available': self.assemblyai_available,
'voice_recording_available': self.assemblyai_available,
'transcription_available': self.assemblyai_available,
'api_key_configured': bool(self.api_keys.get('ASSEMBLYAI_API_KEY')),
'sdk_available': ASSEMBLYAI_AVAILABLE,
'direct_api_available': bool(self.api_keys.get('ASSEMBLYAI_API_KEY'))
}
def _print_status(self):
print("\n๐ค VOICE FEATURES STATUS:")
print(f" AssemblyAI Available: {'โ
' if self.assemblyai_available else 'โ'}")
print(f" Voice Recording: {'โ
' if self.assemblyai_available else 'โ'}")
print(f" Audio Transcription: {'โ
' if self.assemblyai_available else 'โ'}")
print(f" API Key Configured: {'โ
' if self.api_keys.get('ASSEMBLYAI_API_KEY') else 'โ'}")
print()
Comprehensive system for monitoring AssemblyAI feature availability including SDK status, API key configuration, and transcription
capabilities. Provides detailed status reporting for troubleshooting and system health monitoring.
๐ 7. HTTPS Configuration for Microphone Access
python
def create_self_signed_cert():
try:
from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives import serialization
import datetime
import ipaddress
private_key = rsa.generate_private_key(
public_exponent=65537,
key_size=2048,
)
subject = issuer = x509.Name([
x509.NameAttribute(NameOID.COUNTRY_NAME, "US"),
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "Local"),
x509.NameAttribute(NameOID.LOCALITY_NAME, "Local"),
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "AI Learning Platform"),
x509.NameAttribute(NameOID.COMMON_NAME, "localhost"),
])
cert = x509.CertificateBuilder().subject_name(
subject
).issuer_name(
issuer
).public_key(
private_key.public_key()
).serial_number(
x509.random_serial_number()
).not_valid_before(
datetime.datetime.utcnow()
).not_valid_after(
datetime.datetime.utcnow() + datetime.timedelta(days=365)
).add_extension(
x509.SubjectAlternativeName([
x509.DNSName("localhost"),
x509.DNSName("127.0.0.1"),
x509.IPAddress(ipaddress.IPv4Address("127.0.0.1")),
]),
).sign(private_key, hashes.SHA256()) certificate and key
with open("cert.pem", "wb") as f:
f.write(cert.public_bytes(serialization.Encoding.PEM))
with open("key.pem", "wb") as f:.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption()
))
print("โ
Self-signed certificate created")
return True
except Exception as e:
print(f"โ Failed to create certificate: {e}")
return False
Creates self-signed SSL certificates required for browser microphone access. Generates cryptographic certificates for localhost with
proper domain configuration, enabling secure audio recording in web browsers for the educational platform.
Tech Stack Used
Backend
- Python + Flask - For handling sessions and inference
- AssemblyAI - Real-time transcription (Streaming API)
- Groq (LLaMA3-8B) - For instant feedback
Frontend
- JavaScript - Audio streaming + Web Audio API
- HTML/CSS - Minimal, responsive, focused on clarity
๐ญ Final Thoughts
This wasnโt just a submission.
This was a love letter ๐๐ to every shy, nerdy student who ever wished their thoughts could come out clearer.
Itโs funny.
We spend years learning things, but no one ever bothered teaching us how to say them well.
This project is my way of fixing that - with code, care, and a mic.
Would I build more on top of this? Absolutely.
Would I cry if I win? Probably.
Would I still keep improving it if I lose? Without a question. ๐ฅน
๐ซถ Thank You for Listening (Literally)
To the judges, mentors, and every dev reading this โ
Letโs speak better. Letโs build louder.
And maybeโฆ letโs stutter a little less along the way.
๐ค๐
Divya Singh
Thank you for reading till the end
Top comments (21)
Nice!
You don't miss any dev Challenge, do you! ๐
All the best ๐ฅณ
Not 100% of them, i just create multiple submissions for a single challenge mostly ๐
Thank you ๐ฅน
That's a great strategy!
I barely ever get time to submit one project, and that too only on some challenges. ๐
You may like my last submission though. ๐
It seems useful, but is the github repo completely updated?
I'll add some more instructions and polishing, but it already does what is advertised on the post. YES!
I will check it out then ๐
I hope you win this one!
I hope so as well ๐
Thank you ๐
this is really cool ๐ฅ
Thank you for checking it out ๐๐
Nice work! Is there a way to run the above app/demonstrate the functionality in a non-metered environment?
Non- metered as in without the api?
Smart minds always get the right meaning, yes but without the api-key to be precise or with some public demo key....
The project's main feature is listening, understanding and then analysis of it, and feedback for the learner.
It needs these 2 apis , or any 2 ig, for the audio part and the analysis + feedback part.
you are a rockstar, completed 3 challenges and all are awesome.
Great work, liked all of your three challenges. โค๏ธ
Wish you all the very best for these challengesโจโจโจ
Thank you Mr Ninja ๐
Not that awesome, plus it was a last minute rush, but yup, ultimately submitted it all before the deadline.
amazing๐ฅ
Glad you liked it ๐
thanks
Some comments may only be visible to logged-in visitors. Sign in to view all comments.