DEV Community

Fabrikapp
Fabrikapp

Posted on β€’ Edited on

1

TranscribeNow : Live Note-taking and Subtitles

This is a submission for the AssemblyAI Challenge : No More Monkey Business.

Submission for Realtime Audio Prompt

What I Built

For the AssemblyAI Challenge, I developed a real-time audio transcription and note-taking application. This project combines the power of AssemblyAI's Streaming API with a user-friendly interface to provide instant transcription, live note-taking, and AI-assisted content generation.

The application consists of three main components:

  1. A Chrome extension for capturing tab audio, displaying subtitles, and fetching audio from any webpage or microphone
  2. A server-side component for handling WebSocket connections and interacting with AssemblyAI's API
  3. A frontend web application for displaying transcriptions and managing notes. The user will be able to rewrite and generate notes from a recorded session

Demo

Live demo : https://devto-assemblyai-hackaton-frontend-581527456637.us-central1.run.app/

Source Code

Source Code Repository

Screenshots

Note Generation
Review and edit transcribed sessions with additional context and notes

List Your Sessions
AI-assisted note generation based on the transcribed content

Dashboard
The dashboard interface showing live transcription and note-taking stats

Journey

Integrating AssemblyAI's Lemur API was an interesting part of this project:

  1. API Integration: I added server-side actions to interact with Lemur for summarization, question answering, and action item generation. This involved learning the API endpoints and response structures.

  2. User Interface: Incorporating Lemur's features into the frontend required some UI/UX considerations to make the AI-generated content accessible and useful to users.

  3. Learning Curve: Getting familiar with Lemur's capabilities took some time. I experimented with different prompts and parameters to understand how to best utilize the API for our use case.

  4. Added Value: Lemur allowed the application to go beyond simple transcription, offering users more insights from their audio content.

While integrating Lemur had its challenges, it ultimately enhanced the functionality of the application, providing users with AI-powered analysis of their transcribed content.

By integrating these additional tools, I was able to create a more comprehensive and powerful application that goes beyond simple transcription. The Chrome extension allows for seamless audio capture from any web content, while the AI-assisted content generation provides valuable insights and summaries to users, making their note-taking process more efficient and effective.

Throughout this project, I gained valuable experience in working with real-time audio processing, WebSocket communication, and integrating AI capabilities into a web application. The AssemblyAI Streaming API proved to be robust and reliable, enabling me to create a responsive and accurate transcription experience for users.

DEV Challenges are live now!

DEV Challenges Hub

Check out all the ways to participate, hone your skills, and win prizes.

Visit the Challenges Hub

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

πŸ‘‹ Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay