Discussion on: Twilio Hackathon Help Thread

[Michael Lustig - [e-mail]]

Request for async assistance:

Can someone help me out here?

I'm using this library to download a video: github.com/fent/node-ytdl-core

I need to save the video to a file in order to make some modifications. I'm having trouble saving the response of the call to download a video without using the piping operator.

I think the read call is returning a byte array, but can't tell from the method signature. Could someone point me in the right direction here?

  const readable = ytdl(shortVideoUrl);

  await new Promise(res => {
    readable.addListener("readable", () => {
      console.log(`index::20: entering`);
      const foo = readable.read();
      if (!foo) {
        console.log(`index::23: no here`);
      }
      console.log("index::21: foo.length:", foo.length);
      const file = fs.createWriteStream("shortvid.flv");
      file.write(foo);
    });
  });

Alternately, if there is a way to get information about the progress of the write when using the pipe operator, that would be great too. Thanks!

[Phil Nash]

To get info about the progress it looks like you can listen to the progress event on the readable. Something like:

readable.addEventListener('progress', (chunkByteLength, totalBytesDownloaded, totalBytes) => {
  console.log(`Downloaded ${totalBytesDownloaded} out of ${totalBytes}`);
})

I haven't used this library though, so I might be wrong. Hope it helps!

[Michael Lustig - [e-mail]]

Hey Phil thanks a lot for the reply. That definitely works if I wasn't using the pipe operator, but without doing so, I don't know how to create the file from the array of bytes.

At any rate, I actually switched to Python and it's made my life a little easier.

[Phil Nash]

Ah! Because it's a readable stream it will emit the data event with chunks of the data which you can write into your writable stream. Something like this should work:

const file = fs.createWriteStream("shortvid.flv");
const readable = ytdl(shortVideoUrl);
readable.addEventListener('data', chunk => {
  file.write(chunk);
});
readable.addEventListener('progress', (chunkByteLength, totalBytesDownloaded, totalBytes) => {
  console.log(`Downloaded ${totalBytesDownloaded} out of ${totalBytes}`);
})
readable.addEventListener('end', () => {
  file.end();
});

But, if you've switched to Python and you're happy with that, then that's cool too!

[Michael Lustig - [e-mail]]

If only the Twilio dev evang team could answer all of my Javascript queries 🤣

So just for my learning opportunity here - when you call file.write(chunk), it's appending those bytes to the file, and calling file.end() will close that file write stream and result in the video file automagically?

Side note about python: It's been awesome to stand up my hack. The last step of adding a message queue (which I thought would be the hardest) has been the easiest. Gotta love learning!

[Phil Nash]

Yes, that is the gist of it! Using the data and end events of the readable and calling write and end of the writeable is actually the equivalent of using readable.pipe(writeable). But, the documentation suggests you don't want to mix pipe and event based operations and using the progress event would mess that up.

I'm happy to answer JS questions if I can! I just happened to work with other readables/writeables a couple of weeks ago, so it was sort of fresh.

It's good to hear that the Python version has come on nicely for you. I love hackathons for the learning potential!

[Michael Lustig - [e-mail]]

Brilliant! Well I followed you so be expecting some in the future.

Python for whatever reason (or perhaps the competitive nature of the Hackathon) has me more excited about developing than I’ve been in a while, so thanks for that!

And thanks again for your assistance.