Deepgram x DEV Hackathon Help Thread

If you're participating in the Deepgram Hackathon on DEV, we’re so excited to have you joining us! Need some help with your submission or participation? You’re in the right spot.

If you have any questions about how this contest works (ex: due dates, how to post your submission, picking a category/challenge, etc.) the DEV team is here to help you.

How to Use This Thread

If you’ve browsed Deepgram’s documentation but need some help understanding or implementing a feature, @bekahhw from the Deepgram team will be here to jump in and assist you. This thread is a great resource for you whether you’re taking part in the “Build” or “Innovative Ideas” challenge.

Comment below if you need asynchronous assistance with the Deepgram Hackathon on DEV ❤️

Note: Don’t forget that we’re granting a special profile badge to anyone (outside of the DEV and Deepgram teams) who answers a question and to anyone who ASKS a technical question about Deepgram here in the help thread. Additionally, the DEV team will be randomly selecting one person per badge category to receive $50 USD to the Forem Shop! This goes for anyone in the DEV Community – whether you plan to submit an entry or not. For more information, take a look at the official contest rules here. DEV and Deepgram will wait to answer thread questions until we’ve given the community a chance to hop in.

Open Office Hours with Deepgram Devs

Deepgram will also be hosting open “office hours” on their Twitch Stream every Friday at 6:30 PM UTC throughout the contest (March 11, 18, & 25. April 1 & 8). During these streams, Deepgram will be addressing some of the questions asked in this thread and in the live Twitch chat. If they choose a question you posted to the help thread to talk about on the live stream, you'll be entered to win some Deepgram Swag!

---

If you'd like to share an update on the progress you're making on your project or if you'd like to connect with other participants, please do so in the Community Discussion Thread!

Comments

[Santhosh (sandy inspires)]

@michaeljolley
I'm trying to send real-time audio to deebgram and get the transcription but I get this error while doing so.
I'm using pyaudio to send the binary data.

input_audio = stream_in.read(3200)
await ws.send(input_audio)

DEBUG:websockets.client:< CLOSE 1008 (policy violation) DATA-0000 [11 bytes]

[Michael Jolley]

Hi @sandy_codes_py, I'm not a Python pro, but I can hook you up with @tonyasims. She's an amazing pythonista and one of Deepgram's Developer Advocates.

One thing that might help, is there a GitHub repo where we can review the full block of code?

[Santhosh (sandy inspires)]

Please check here
I've used api reference code from the deepgram docs

[Michael Jolley]

Sweet. Thanks for that. Tonya is out of the office today, but she'll likely respond Monday.

Mental note: I should really start playing with Python more so I can help more. 🙂

[Santhosh (sandy inspires)]

That would do!
I did find a workaround it by using her post "Live Transcription With Python and Flask" but that's not really needed here.
I just want to run it locally to do something cool.
Yay, even I've to learn about async and how it works.

[Michael Jolley]

Great! I think she wrote several posts like that: "using Flask," "using Django," "using FastAPI," and more.

And yeah! for learning! 🎉🎉

[Tonya Sims]

Hi @sandy_codes_py ! Happy Monday! I'm sorry to hear you were having some trouble. Were you able to figure out the issue? Please let me know if you still need some help and we can work through it together.

[Santhosh (sandy inspires)]

Nope, I followed your Flask repo and did that same. But I really want to run that in just Python and not on Flask. I do have limited knowledge on asyncio. Will be learning that soon. I've shared the error and the complete code I'm using the same thread.

[Tonya Sims]

Ok, so if I understand correctly you follwed the Flask example in the tutorial but still having some issues? Is the issue still with PyAudio?

Also, is this the error message you received? (I want to make sure this is the correct error):

input_audio = stream_in.read(3200)
await ws.send(input_audio)

DEBUG:websockets.client:< CLOSE 1008 (policy violation) DATA-0000 [11 bytes]

[Santhosh (sandy inspires)]

Sorry to misled you!
Actually I wanted to directly send the audio feed using PyAudio to the Deepgram websocket (that's when the above error is occurring).
But I found your Flask tutorial and used that instead which worked in the first go.

The complete code I used can be found here.

[Tonya Sims]

Oh nice! Good to hear you found a solution with the tutorial 😄.

About asyncio, yea, I totally understand it can be very confusing. It took me awhile to wrap my head around it. How do you plan on learning asyncio? Tutorials? Blog posts? Videos? Something else?

[Santhosh (sandy inspires)]

Gonna read through this for a while and try some hands-on stuff.
docs.python.org/3/library/asyncio....
I'll try to make a tutorial here once I get a good grasp.

[Tonya Sims]

Wonderful! Make sure to let me know when it's published so I can read it :)

[Santhosh (sandy inspires)]

You got it!

[FrankPohl]

Hi,
I would like to participate with a .NET solution using your new .NET SDK.
This should use the audio from the microphone to implement a voice dialog within the app (kind of a bot and some speech control for the app).
Unfortunately, I could not find any sample how to connect the Microphone stream with the CreateLiveTranscriptionClient.
Can you give me some hint show to connect the mic audio with the SendData function?
Thanks Frank

[Michael Jolley]

Hi Frank, do you have a stream of audio to send? As in, have you already captured audio from the mic?

[FrankPohl]

Hi Michael,
I can get the audio in from the mic and write it to a file with the help of the NAudio package..
There you can see the code that writes the input to a file. It is in the event handler OnDataAvailable but as a comment.
But this is the same method that has the call SendData to send data to the deepgram service.
I have posted my code in this repository github.com/FrankPohl/HealthAssista...
This is a MAUI app because besides console apps, this is the only platform with .NET 6 support.
I use this because my plan was to build an app to help handicapped people to put in data like their blood pressure with speech. Therefore I need a UI and a permanent connection from the microphone to the recognition service.

Btw., you should have mentioned in your sample code with the logging what nuget packages you have used. Took me some time to figure out why "AddSerilog" did not work.

[FrankPohl]

I can add more information about the format from the Microphone stream.
Samplerate: 48000
Encoding: IeeeFloat
Bits: 32
Channels: 2
Blockalign: 8
Bytes per Second: 384000

Must this be converted or should it be processable from the interface?

I added the file test.wav to the repository. This file was recorded from my microphone.
I also added a function to send a file for conversion to the deepgram service.
Trying to convert this raise error 400 (Bad Request).
I hope someone can come up with a solution so that I can continue to work on my contribution to the hackathon.

[Michael Jolley]

Thanks @fp. I've been playing with it yesterday and realized it was IeeeFloat. The API doesn't support that format. You'll need to convert it to something else. I noticed in your request you're telling the API you're sending Linear16. That's fine once the raw audio is converted to that.

[FrankPohl]

Hello Michael,
I made a conversion for my input.
I do not get an error when I use the CreateLiveTranscriptionClient but I do not get any result from deepgram but I cannot find out what the problem. Maybe you can have a look? I updated the github project witht he latest code samples.
If I try to convert a file that is PCM with a sample rate of 16000 I get an error from the SDK. The file is TestConverted-16000-Pcm-2.wav and is a recorded and converted audio sample from me.
Would be nice if you could help me with that again.

Frank

[Michael Jolley]

I'm looking at this tonight and will let you know something soon!

[Michael Jolley]

Okay. I created a quick .NET 6 Console app and have it working based on a slightly modified version of your code. You can find it at gist.github.com/MichaelJolley/b52f...

Hope that helps!

[FrankPohl]

Thanks for your help Michael.
What a shame to bother you with such a stupid error in my code.
I would like to publish that program as an example on Git, or will you do that?

[Michael Jolley]

No way @fp! It took me a while to figure it out so don't feel bad at all. I'm glad it's working for you now.

You can certainly publish it if you'd like.

[Dhravya]

I want to participate, but, the post says that it's Open only to 18+.

So, does that mean I'm not eligible ? :(

[BekahHW]

I don't think it means that you aren't eligible for badges for helping. So if you contribute in the Help and Community threads, I think that means you can still earn badges.

[Dhravya]

i have a couple ideas, if i just write a post with he ideas, would I be eligible? (I could even execute them)

[Jonah Lawrence]

The post seems to imply that all parts of the Hackathon are 18+ to participate. It is likely there for legal reasons, and I would think that if you should be able to participate if your parents approve.

[Gracie Gregory (she/her)]

Hello! Unfortunately, DEV hackathons are currently only open to community members 18 years of age and older. We would absolutely love to open this up to younger community members, but we are legally unable to at this time. We revisit the topic frequently as a team and intend to open up our challenges to younger folks if and when it's possible for us to do so. I'm so sorry to bear this news. We really value having all of you as part of DEV.

[mainrs]

It's normal for contests to be 18+ due to prizes and legal ruling.
You can ask the dev people if it's OK if you submit but are not considered as a potential winner.

[Dhravya]

Yeah, I've already submitted my project. It's completely fine if I don't win, or win and not get prizes. atleast I got to learn stuff, that's all that matters

[Michael Jolley]

That's a great attitude @dhravya! I'm REALLY looking forward to checking out your submission.

[Dhravya]

Hi there! Thanks a lot!
Here's the post dev.to/dhravya/deepsubtitles-gener...

[Nabil Alamin]

Well that was fast, great job 👌👌

[Dhravya]

thanks a lot!

[Eli Ostreicher]

Having a little bit of a hard time figuring out the final part of the listen endpoint.
1) In a local environment, how would I go about sending the empty binary message to the server?
2) Upon ending, where is the final transcript JSON now?

Thank you.

[Michael Jolley]

Great question. It depends on how you're hitting the API. If you are using the Node SDK you'd use the finish function, as in:

const deepgramSocket = deepgram.transcription.live({ punctuate: true });
deepgramSocket.finish();

If you're communicating with the WebSocket directly with JavaScript you can send a new Uint8Array, as in:

socket.send(new Uint8Array(0));

When the Deepgram API receives that it will finish transcribing the audio, send a final transcript, and then close the WebSocket connection.

[Eli Ostreicher]

Appreciate the response.
What about good old Python please?

P.S. Great Twitch btw, really enjoyed it.

[Michael Jolley]

Thanks!

For the Python SDK:

deepgramLive = await deepgram.transcription.live()
await deepgramLive.finish()

Against the WebSocket without the SDK would be something like:

await socket.send(b'')

[BekahHW]

Thanks for coming to the stream today!

[tq-bit]

Hello guys,

I'm using the Node.js SDK to stream an audio file from a web application. I'm using Typescript and was wondering why I receive the following ts error here.

Do I really need the version property here? Could anything unexpected happen if I don't?

My current code looks something like this (and works file if I set @ts-ignore on top of it):

import { Deepgram } from '@deepgram/sdk';

...

export default class Transcriber {

   ...

  public translateFromLocalFile = async () => {
    const streamSource = {
      stream: fs.createReadStream(this.filePath),
      mimetype: this.mimeType,
    };

    const response = await this.deepgram.transcription.preRecorded(streamSource, {
      punctuate: true,
    });

    const transscript = this.getHighestRatedTranscript(response)

    return transscript;
  };
  ...
}

...

[Michael Jolley]

Great find. That's a bug in our SDK. version is an optional parameter.

[Michael Jolley]

New release is on NPM now with that fix. Thanks for catching & reporting it.

[tq-bit]

That was quick. Thanks :-)

[Rutam Prita Mishra]

Hello @michaeljolley and DG Team 👋

I just wanted to know do we have any examples where we are writing the transcripts to a separate output file instead of showing the complete json on the console. And by transcript, I mean the only part in the json output that contains the text sentences.

Moreover, let me know how to make sure we scan through the complete length of the audio we are passing instead of just a part of it.

Thanks in advance.

[Michael Jolley]

Hiya @21rutam! Good question. Let me make sure I give you the right answer.

First, are you using one of our SDKs? If so, which one?
Second, are you trying to save the whole payload or just the words?

[Rutam Prita Mishra]

Yeah, I am using NodeJS to do the job. I want to just save the transcript part of the response payload. And do let me know if we can define the duration of the audio for scanning (Complete audio or Just for a timeframe)
CC: @michaeljolley

[Michael Jolley]

You can't define the duration. The API will try to transcribe the entire audio file every time, not just a section.

Using the Node SDK, you could send your request with the utterances feature turned on. (e.g. utterances:true)

Then, when the transcription comes back you can use the .toSRT() or .toWebVTT() functions to generate a text based version of the transcript with timestamps. Then you'd want to save it locally using fs.

Example:

const { Deepgram } = require('@deepgram/sdk');
const fs = require('fs');

const deepgram = new Deepgram(DEEPGRAM_API_KEY)
const audioSource = { url: URL_OF_FILE };

deepgram.transcription.preRecorded(audioSource, {
  punctuate:  true,
  utterances: true,
  // other options are available
})
.then((response) => {
  const srtTranscript = response.toSRT();

  fs.writeFile(FILENAME_TO_SAVE, srtTranscript, function (err) {
    if (err) {
      return console.log(err);
    }
    console.log("The file was saved!");
  });
})
.catch((err) => {
  console.log(err);
});

[Rutam Prita Mishra]

That was real quick. But I don't really want it like a subtitles file. Rather I just want the transcript text to be saved to the file and not anything else. I am talking about the sentences in that transcript: part (the one marked in purple).
CC: @michaeljolley

[Michael Jolley]

You could just use the transcript property itself:

const { Deepgram } = require('@deepgram/sdk');
const fs = require('fs');

const deepgram = new Deepgram(DEEPGRAM_API_KEY)
const audioSource = { url: URL_OF_FILE };

deepgram.transcription.preRecorded(audioSource, {
  punctuate:  true,
  // other options are available
})
.then((response) => {
  const srtTranscript =response.results.channels[0].alternatives[0].transcript;

  fs.writeFile(FILENAME_TO_SAVE, srtTranscript, function (err) {
    if (err) {
      return console.log(err);
    }
    console.log("The file was saved!");
  });
})
.catch((err) => {
  console.log(err);
});

[Rutam Prita Mishra]

Thanks a bunch @michaeljolley . You rock 🙌🚀

[FrankPohl]

Hi,
I'm using the DeepGram standard recognition service without an additional model.
I have a problem with the recognition of "weight" and "wait". Often "wait" is recognized instead of "weight". Is there a way to enhance the probability for "Weight" because in the use case it is true for 99 percent of the cases?
Or is this just a problem of me not being a native speaker?
Frank

[Michael Jolley]

That's an amazing question @fp! You can add some "weight" to the word "weight". (See what I did there. 😁)

You'd want to use the keywords feature. If I remember correctly, you're using the .NET SDK, so here's how you'd set that up:

var options = new LiveTranscriptionOptions()
{
    Punctuate = false,
    Diarize = false,
    Numerals = true,
    Encoding = Deepgram.Common.AudioEncoding.Linear16,
    Language = "en-US",
    InterimResults = true,
    SampleRate = 44100,
    Keywords = new string[1] {"weight"}
};

[FrankPohl]

Thanks @michaeljolley that's exactly what I was looking. Unfortunately this raises :
Deepgram Error: Unexpected character encountered while parsing value: [. Path 'keywords', line 1, position 82.
when the StartConnectionAsync method is called.
The error is raised in line 44 in Helpers.cs
t = JsonConvert.DeserializeObject>(json);

[Mahmudul Hasan]

I was treid to make a Live speech Transcript project, but failed due to unknown error. Can someone please help me on that?

This show in the browser console:
This is error

Here is my GitHub link: github.com/mhasanmeet/DEEPGRAM-liv...

[Michael Jolley]

Nice! I can help with that.

It looks like the credentials you're sending are incorrect.

That code looks like you're sending the API Key Id rather than the API Key. When you create an API key, be sure you're copying the key itself. You should see a screen like below. You can click that copy icon to copy the actual key. Be sure to copy it, because you won't be able to see it again for security purposes.

[Mahmudul Hasan]

Thanks Micheal, I'm gonna try it again. I'm really exicited about this Deepgram project. I have so much idea, I will implement one by one. And thanks, you guys already made some awesome demo projects, it really helpful!

[Michael Jolley]

Awesome! I can't wait to see what you build. Be sure to reach out if I can help!

[Minsu]

Hi! First of all, Love the idea of this hackathon! Thanks for hosting, Deepgram :)

I have question about real-time transcript with deepgram(node.js SDK).

I tried this tutorial below
developers.deepgram.com/documentat...

I am wondering if there is a way to use 'opus stream' audio instead of 'url' here in the tutorial for transcription??

Thank you!

[Michael Jolley]

Solid question. Normally you wouldn't want to send a stream from a URL in, you'd be accessing a microphone and streaming that audio in. I haven't tried opus stream specifically, but you can certainly try it. Is there a reason you need opus stream? That seems like an unusual format for live streaming.

[Minsu]

Thanks for reply Michael! yeah We use opus because we are building a real time transcription discord bot for deaf gamers! And discord.js uses opus as a format… My teammates gave up in the middle so I don’t think I can finish it by 11th but still wants to get this one done :)

[Michael Jolley]

That's amazing! So I think you may can stream that in but you'd want to make sure you specify the encoding, sample rate, etc. in your request.

[sonu0702]

I created Deepgram account. But How do i
Select one of the following four categories

1. Accessibility Advocates
2. Analytics Ambassadors
3. Gram Gamers
4. Wacky Wildcards

I don't see any of this options on the Deepgram website

[Michael Jolley]

The instructions said "Select one of following categories," but it isn't clear that you're really selecting it in your mind. You're deciding what category you want to build a project for. When you create a submission you'll use this template to create your submission post. That template has a section where you'll enter what category you're submitting your project for.

[sonu0702]

Thanks, It's helpful

[FrankPohl]

@michaeljolley I create a sample to send audio directly to deepgram from the microphone.
I have created a repository for this sample here github.com/FrankPohl/DeepGram.NETS...
But there is one thing I do not understand. I resample the audio input to PCM with a sample rate of 16000. But in the deepgram options is 44100 given as a sample rate. If I change that to 16000 I do not get a transcription. Why is that?

[Michael Jolley]

That's a REALLY good question @fp! That code isn't resampling, it's converting it from 32-bit to 16-bit. IeeeFloat is a 32-bit format. We're basically converting C# long into C# short. This blog post does a good job at describing the differences in the two.

[FrankPohl]

@michaeljolley Which code is not resampling? The code in my example on Github converts from to short, that's right. But in a second step it does resampling because I'm averaging 3 consecutive input values into one output value. I thing this means that the sampling rate is reduced from 48000 to 16000. The wav file that is written in parallel has this sampling rate and sounds alright.

[Nikunj R. Prajapati]

I have one interesting question here,
i just started exploring deepgram, i wonder how deepgram is different than other STT Python libraries out there ? as there are so many opensource python libraries are available for example github.com/mozilla/DeepSpeech !

[Tonya Sims]

Hi @leo1612d! Thank you for your question. I'm not as familiar with DeepSpeech, the mozilla library you posted so I can only speak to Deepgram.

At Deepgram our mission is we believe every voice should be heard and understood.

A major advantage Deepgram has over other STT libraries is that we're the only provider that uses End-to-End Deep Learning. This means that you'll get improved accuracy with training, an easier to use and integrated API and is priced to use at scale.

[Vinícius Gajo]

How do you guys at Deepgram deal with the data provided by users? Is there any document describing it?

[Jonah Lawrence]

There is information about the data collection in the Terms of Service Agreement, the Privacy Notice, and Data Privacy Policy.

From what I can tell, uploaded data is collected and stored, measures are in place to protect the data, and customers will receive an email or message about any security notices.

You may upload, or transmit files, ... User Content, including personal information you may have included, is stored and collected as part of the Service...

...our customers may upload data to our Service ... which may include personal information ... We collect and process Customer Data as needed to provide our Services to our customers in accordance with our customer agreements.

We will retain your personal information for the length of time needed to fulfill our business purposes unless otherwise required or permitted by law. Any Customer Data that we have access to shall be retained, stored, and deleted according to our agreement with our business customer.

Deepgram maintains industry-standard physical, technical, and administrative safeguards in order to protect Content and Training Data

Note: I am not a member of the Deepgram team, so any clarifications should be directed towards them.

[Vinícius Gajo]

Got it, thanks for the answer.

[Kevin Lewis]

Please read through the Terms of Services and the Privacy Policy on questions related to data retention. In short, Deepgram stores and processes a small percentage of the data for the purposes of improving services and/debugging debugging client requests. If you have further inquiries, including one-off requests for data deletion, please reach out to the Deepgram customer success team at [e-mail].

In the case of a breach, end users will be notified within 72 hours of breach confirmation.

[Kevin Lewis]

What do you mean by 'deal'? Just making sure we can answer your question correctly :)

[Vinícius Gajo]

I want to understand if you guys store this kind of information that is given by your users (input audio). If it's stored, how you manage to keep it safe from public access. And if, for some reason, this data leak, what you guys will do, for example, alert people contacting through e-mail.

[jzombie]

I'd be curious to know where to look for any data retention policies concerning those of accidental streaming transcriptions when someone forgot to turn off their mic.

[Mr. Bud]

Hey there @michaeljolley, hope you all are doing great!

Just curious about the Article Template DEV has presented. I think I can change the heading and the other things according to need, and of course, including the everything mentioned.

Am I right? Thanks!

[Michael Jolley]

You're 100% right!

[Santhosh (sandy inspires)]

Can we have multiple submission for this hackathon?

[Dhravya]

The post says yes and even encourages multiple submissions, so yes i guess

[BekahHW]

Yes! We'd love to see you get as many chances as possible to win!