VSChool Web App with Flask, Vue, Mongo, Deepgram and Dropbox

—

Overview of My Submission

Due to Covid19 global pandemic many tasks are now done online including educational tasks such as content delivery. Many schools now have some of the content delivered through platforms like Google Classroom, Microsoft Teams, etc. with teachers/instructors deliver content to students in different formats such as audio, videos or texts.

One of the challenges I noticed is that some of teachers' pre-recorded content might have some parts which are not clear to their audience. This might cause students to contact their teachers for clarity or ignore those parts which may result as a distraction.

This challenge can be solved using Deep gram's powerful weapons which was VSChool's motivation. VSChool web app is a simple platform which allows users to upload an audio file, generate transcriptions which are used to create subtitles for the video that can be played by users.

System Flow

• user can create a subject and topics

• subject have many topics

• user can also upload an audio (MP3) for each and every topic

• the system will generate transcriptions and a subtitles file (WEBVTT), the complete process will be explained later

• it will then upload both MP3 and VTT files to Dropbox

• it will get MP3 and VTT shared links and save them in the database

• web app will fetch topics with shared links and render respective media

• MP3 is rendered as a video with its subtitles are obtained from VTT file

• hence, users will watch a video with subtitles generated by the system

The process of generating subtitles

• app sends a POST request to a RESTful API with an MP3 audio and topic_id in the payload

• API will rename the audio file to a topic_id as a new file name

• it will also create a new directory named after a topic_id and stored in the media_files directory

• the audio file will be saved in this new created directory

• the audio file will then sent to Deepgram for transcription with utterances set to True

• JSON response from Deepgram is written in a WEBVTT file for video subtitles

• VTT file is named after topic_id too

• it is then saved in a directory created earlier (the one named after topic_id)

• the directory is then compressed to a ZIP file, in same parent folder
  -then the original directory is deleted to only left with a ZIP file

• ZIP file is uploaded to Dropbox

• Dropbox destination folder is configured to automatically extract new ZIP files to become folders

• the system will sleep for 10 seconds to wait for Dropbox to extract ZIP files

• it will then send two requests to create shared links for MP3 and VTT files in new folders in Dropbox

• the system will save these to created shared links to the respective topic

• these shared links will then be used by the App to play the video with subtitles

How to Set up the environment

API:

• Add '.env' file with credentials as follows:

• DEEPGRAM_API_KEY=""

• DROPBOX_ACCESS_TOKEN = ""

• DROPBOX_APP_KEY = ""

• DATABASEUSER=""

• DATABASEPASSWORD=""

• CLUSTER=""

• DATABASE=""

• HOST=""

• JWT_SECRET_KEY = ""
  Please fill blank "" with respective values

• Create Python virtual environment in project's folder using command 'python -m venv myenv'

• Activate virtual environment using command 'ENV_NAME\scripts\activate'

• Install required libraries in 'requirements.txt' file using command 'pip install -r requirements.txt'

• Start the app using command 'flask run'

App:

• Prerequisites @vue/cli 4.5.12 and npm

• Run command 'npm install' to install all required libraries

• Run command 'npm run serve' to start the app

Demo Video

• https://youtu.be/_93XB16LWUI

Submission Category:

Accessibility Advocates

Link to Code on GitHub

• Vue App Link: https://github.com/ValentineSean/v-school-app

-Flask API Link: https://github.com/ValentineSean/v-school-api