DEV Community

Cover image for How to Create a YouTube Downloader with Node.js and React.js
Prathamesh More
Prathamesh More

Posted on

How to Create a YouTube Downloader with Node.js and React.js

The basic flow of the app:

  1. User will provide a YouTube video link
  2. The backend server will push this video link in the queue to process the download
  3. When the job popped from the queue for processing, the backend emits the event for the client
  4. The client listens to the event and shows appropriate messages
  5. Users will able to download videos from a server

We will use Socket.io for the emitting events and processing and handling jobs will use the Bull package.

Let's begin,

Install required software and packages on your local machine

Software Requirements:

  1. Node.js - Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine.
  2. Postman - A collaboration platform for API development.

Require packages:

Backend packages:

npm i typescript express mongoose cors express-validator mongoose morgan socket.io ytdl-core bull dotenv

Frontend packages:

npm i axios js-file-download socket.io-client

Setup Backend:

We will be using the MongoDB database so make sure you install it locally or you can use free cloud service from MongoDB.

Set up Redis database with Upstash:

Upstash is a serverless database for Redis. With servers/instances, you usually pay per hour or at a fixed price. With serverless, you pay-per-request.

This means you're not charged when the database isn't in use. Upstash configures and manages the database for you.

Start by creating an account on Upstash.

Now set up the Redis database instance

Create new database

Connect to your database

Let's initialize TypeScript based Node.js project,

tsc --init
then do
npm init -y
Enter fullscreen mode Exit fullscreen mode

Root Directory Structure

Don't forget to add .env file and its content.

.env file

Create a new src directory in the root directory of the project as shown in the above image.

Create a simple server and connect to the local or remote MongoDB database:

import { config } from "dotenv";
config();
import http from "http";
import express, { Request, Response } from "express";
import { Server } from "socket.io";
import mongoose from "mongoose";
import cors from "cors";
import path from "path";
import morgan from "morgan";
import { SocketInit } from "./socket.io";

const app = express();

const server = http.createServer(app);

export const io = new Server(server, {
  cors: { origin: "*" },
});

new SocketInit(io);

mongoose
  .connect(process.env.MONGO_DB, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  })
  .then(() => {
    console.log("Connected to database");
  })
  .catch((error) => {
    throw error;
  });

app.use(morgan("dev"));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(cors());

app.get("/", (req: Request, res: Response) => {
  res.status(200).send('ok')
});

server.listen(3000, () => {
  console.log("Server running up 3000");
});
Enter fullscreen mode Exit fullscreen mode

Now, create a mongoose model for store video metadata, this file will reside in src/models.

import mongoose from "mongoose";

export interface VideoDoc extends mongoose.Document {
  title: string;
  file: string;
  thumbnail: string;
}

const videoSchema = new mongoose.Schema(
  {
    title: {
      type: String,
      required: true,
    },
    file: {
      type: String,
      required: true,
    },
    thumbnail: {
      type: String,
    },
  },
  { timestamps: true }
);

export const Video = mongoose.model<VideoDoc>("video", videoSchema);
Enter fullscreen mode Exit fullscreen mode

REST APIs

REST APIs Routes
1. GET => /api/donwloads => Get all downloads
2. GET => /api/donwloads/:id => Get a single download
3. POST => /api/downloads => Push new download
4. DELETE => /api/downloads/:id => Remove a single download
5. GET => /api/downloads/:id/downloadfile => Download a single file
Enter fullscreen mode Exit fullscreen mode

Let's implement controllers and routes for APIs,

import express, { Request, Response, NextFunction } from "express";
import fs from "fs/promises";
import { Video } from "../models/video";
const downloadsRouter = express.Router();

downloadsRouter.get(
  "/api/downloads",
  async (req: Request, res: Response, next: NextFunction) => {
    const videos = await Video.find().sort({ createdAt: -1 });
    res.status(200).send(videos);
  }
);

downloadsRouter.get(
  "/api/downloads/:id/downloadfile",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;
    const video = await Video.findById(id);

    if (!video) {
      res.status(404).send("Video not found");
    }
    const { file } = video;

    res.status(200).download(file);
  }
);

downloadsRouter.post(
  "/api/downloads",
  body("youtubeUrl").isURL(),
  async (req: Request, res: Response, next: NextFunction) => {
    //Will implement
  }
);

downloadsRouter.delete(
  "/api/downloads/:id",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;

    const video = await Video.findByIdAndDelete(id);

    if (video) {
      await fs.unlink(video.file!);
    }
    res.status(200).send(video);
  }
);

export { downloadsRouter };
Enter fullscreen mode Exit fullscreen mode

Now here comes the most important task,

This is section will implement a download queue using Bull Queue.

However every queue instance will require new Redis connections.

This queue will process all downloads one by one.

On each job process, we are emitting events for the client.

import Bull from "bull";
import ytdl from "ytdl-core";
import fs from "fs";
import { Video } from "../models/video";
import { Events } from "../utils";
import { SocketInit } from "../socket.io";

const downloadQueue = new Bull("download queue", {
  redis: {
    host: process.env.REDIS_HOST!,
    port: parseInt(process.env.REDIS_PORT!),
    password: process.env.REDIS_PASSWORD
  },
});

downloadQueue.process((job, done) => {
  return new Promise(async (resolve, reject) => {
    const { youtubeUrl } = job.data;

    //Get singleton instance
    const socket = SocketInit.getInstance();

    const info = await ytdl.getBasicInfo(youtubeUrl);

    console.log(info.videoDetails.thumbnails[0].url);

    const thumbnail = info.videoDetails.thumbnails[0].url;

    //Appending some randome string at the end of file name so it should be unique while storing on server's disk
    const title =
      info.videoDetails.title +
      " by " +
      info.videoDetails.author.name +
      "-" +
      new Date().getTime().toString();

    ytdl(youtubeUrl)
      .pipe(fs.createWriteStream(`${process.cwd()}/downloads/${title}.mp4`))
      .on("finish", async () => {
        socket.publishEvent(Events.VIDEO_DOWNLOADED, title);

        const file = `${process.cwd()}/downloads/${title}.mp4`;

        const video = new Video({
          title,
          file,
          thumbnail,
        });

        await video.save();

        done();

        resolve({ title });
      })
      .on("ready", () => {
        socket.publishEvent(Events.VIDEO_STARTED, title);
      })
      .on("error", (error) => {
        socket.publishEvent(Events.VIDEO_ERROR, error);
        done(error);
        reject(error);
      });
  });
});

export { downloadQueue };
Enter fullscreen mode Exit fullscreen mode
export enum Events {
  VIDEO_DOWNLOADED = "VIDEO_DOWNLOADED",
  VIDEO_STARTED = "VIDEO_STARTED",
  VIDEO_ERROR = "VIDEO_ERROR",
}
Enter fullscreen mode Exit fullscreen mode

Whenever a user tries to download a video, we first push that job i.e. link in download queue.

Then we request for Socket.io instance and video's metadata like title and thumbnail.

//Get existing instance
const socket = SocketInit.getInstance();
const info = await ytdl.getBasicInfo(youtubeUrl);
const thumbnail = info.videoDetails.thumbnails[0].url;
Enter fullscreen mode Exit fullscreen mode

Using ytdl package, we start downloading the file and storing it in a directory called downloads in the root of the project.

When the download starts we emit event VIDEO_STARTED with a title as data.

When the download completes we emit event VIDEO_DOWNLOADED.

When the download gets failed due to some reason like private video or copyright content, we emit event VIDEO_ERROR.

Now import this queue module in the controller, also we added some validation on the request body.

import express, { Request, Response, NextFunction } from "express";
import fs from "fs/promises";
import { body, validationResult } from "express-validator";
import { downloadQueue } from "../queues/download-queue";
import { Video } from "../models/video";
const downloadsRouter = express.Router();

downloadsRouter.get(
  "/api/downloads",
  async (req: Request, res: Response, next: NextFunction) => {
    const videos = await Video.find().sort({ createdAt: -1 });
    res.status(200).send(videos);
  }
);

downloadsRouter.get(
  "/api/downloads/:id/downloadfile",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;
    const video = await Video.findById(id);

    if (!video) {
      res.status(404).send("Video not found");
    }
    const { file } = video;

    res.status(200).download(file);
  }
);

downloadsRouter.post(
  "/api/downloads",
  body("youtubeUrl").isURL(),
  async (req: Request, res: Response, next: NextFunction) => {
    try {
      const errors = validationResult(req);
      if (!errors.isEmpty()) {
        return res.status(400).json({ errors: errors.array() });
      }
      const { youtubeUrl } = req.body;
      await downloadQueue.add({ youtubeUrl });
      res.status(200).send("Downloading");
    } catch (error) {
      throw error;
    }
  }
);

downloadsRouter.delete(
  "/api/downloads/:id",
  async (req: Request, res: Response, next: NextFunction) => {
    const { id } = req.params;

    const video = await Video.findByIdAndDelete(id);


    if (video) {
      await fs.unlink(video.file!);
    }
    res.status(200).send(video);
  }
);

export { downloadsRouter };
Enter fullscreen mode Exit fullscreen mode

Finally we can add this controller in server.ts file,

import { config } from "dotenv";
config();
import http from "http";
import express, { Request, Response } from "express";
import { Server } from "socket.io";
import mongoose from "mongoose";
import cors from "cors";
import path from "path";
import morgan from "morgan";
import { SocketInit } from "./socket.io";
import { downloadsRouter } from "./routes/downloads";

const app = express();

const server = http.createServer(app);

export const io = new Server(server, {
  cors: { origin: "*" },
});

new SocketInit(io);

mongoose
  .connect(process.env.MONGO_DB, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
  })
  .then(() => {
    console.log("Connected to database");
  })
  .catch((error) => {
    throw error;
  });

app.use(morgan("dev"));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.set("view engine", "ejs");
app.use(express.static(path.join(__dirname, "views")));
app.use(cors());
app.use(downloadsRouter);

app.get("/", (req: Request, res: Response) => {
  res.render("index");
});

server.listen(3000, () => {
  console.log("Server running up 3000");
});
Enter fullscreen mode Exit fullscreen mode

Finally, change scripts in package.json

"scripts": {   
   "start": "ts-node src/server.ts",
   "dev": "ts-node-dev src/server.ts"
}
Enter fullscreen mode Exit fullscreen mode

Now test with Postman,

POST => /api/downloads

Push new download

GET => /api/downloads

Request all videos

Setup Frontend:

Create boilerplate code for React by running the following command:

npx create-react-app fronend && cd frontend

The folder structure looks like after running the command,

Folder Structure for React.js app

Then we just added Components directory, we have there three components

Three components

Now add Bootstrap for UI:

Bootstrap

Design basic nav bar:

import React from "react";

export default function Navbar() {
  return (
    <header class="pb-3 mb-4 border-bottom">
      <a
        href="/"
        class="d-flex align-items-center text-dark text-decoration-none"
      >
        <svg
          xmlns="http://www.w3.org/2000/svg"
          width="50"
          height="50"
          fill="currentColor"
          class="bi bi-youtube"
          viewBox="0 0 16 16"
        >
          <path d="M8.051 1.999h.089c.822.003 4.987.033 6.11.335a2.01 2.01 0 0 1 1.415 1.42c.101.38.172.883.22 1.402l.01.104.022.26.008.104c.065.914.073 1.77.074 1.957v.075c-.001.194-.01 1.108-.082 2.06l-.008.105-.009.104c-.05.572-.124 1.14-.235 1.558a2.007 2.007 0 0 1-1.415 1.42c-1.16.312-5.569.334-6.18.335h-.142c-.309 0-1.587-.006-2.927-.052l-.17-.006-.087-.004-.171-.007-.171-.007c-1.11-.049-2.167-.128-2.654-.26a2.007 2.007 0 0 1-1.415-1.419c-.111-.417-.185-.986-.235-1.558L.09 9.82l-.008-.104A31.4 31.4 0 0 1 0 7.68v-.123c.002-.215.01-.958.064-1.778l.007-.103.003-.052.008-.104.022-.26.01-.104c.048-.519.119-1.023.22-1.402a2.007 2.007 0 0 1 1.415-1.42c.487-.13 1.544-.21 2.654-.26l.17-.007.172-.006.086-.003.171-.007A99.788 99.788 0 0 1 7.858 2h.193zM6.4 5.209v4.818l4.157-2.408L6.4 5.209z" />
        </svg>
        <span className="fs-4">YouTube Downloader</span>
      </a>
    </header>
  );
}
Enter fullscreen mode Exit fullscreen mode

Now integrate all download APIs in Home.js component,

Here we making connection with server using socketio-client for events and aslo making HTTP request for data.

import React, { useEffect, useState } from "react";
import axios from "axios";
import toast, { Toaster } from "react-hot-toast";
import { io } from "socket.io-client";
import Videos from "./Videos";

const notify = (msg, { success }) => {
  if (success) {
    return toast.success(msg);
  }
  return toast.error(msg);
};

const socket = io("http://localhost:3000/");

export default function Home() {
  const [videos, setVideos] = useState([]);

  useEffect(() => {
    socket.on("VIDEO_DOWNLOADED", (data) => {
      notify(`${data} Downloaded`, { success: true });
      window.location.reload();
    });

    socket.on("VIDEO_STARTED", (data) => {
      notify(`Download Started ${data}`, { success: true });
    });

    axios
      .get("http://localhost:3000/api/downloads")
      .then((res) => {
        setVideos(res.data);
      })
      .catch((error) => {
        console.log(error);
      });
  }, []);

  const downloadVideo = (event) => {
    event.preventDefault();

    const youtubeUrl = event.target.elements.youtubeUrl.value;

    axios
      .post("http://localhost:3000/api/downloads", { youtubeUrl })
      .then((res) => {
        notify("Fetching video details...", { success: true });
      })
      .catch((error) => {
        notify("Something went wrong", { success: false });
      });
  };

  return (
    <div>
      <div class="p-5 mb-4 bg-light rounded-3">
        <div class="container-fluid py-5">
          <h1 class="display-5 fw-bold">
            Download your favorite Youtube videos
          </h1>
        </div>
        <form onSubmit={downloadVideo}>
          <div>
            <label for="youtubeUrl" class="form-label">
              Enter link
            </label>
            <input type="url" id="youtubeUrl" class="form-control" required />
            <div id="urlHelpBlock" class="form-text">
              E.g. https://www.youtube.com/watch?v=PCicKydX5GE
            </div>
            <br />
            <button type="submit" class="btn btn-primary btn-lg">
              Download
            </button>
            <Toaster />
          </div>
        </form>
      </div>
      <h3>Downloaded videos</h3>
      <div style={{ margin: 10 }} className="row">
        {videos.map((video) => {
          return <Videos video={video} />;
        })}
      </div>
    </div>
  );
}
Enter fullscreen mode Exit fullscreen mode

Now let's implement Video.js component to render each single video and related operation,

import axios from "axios";
import React from "react";
const FileDownload = require("js-file-download");

export default function VideoDownloader(props) {
  console.log(props);
  const { video } = props;
  const { _id, title, thumbnail } = video;

  const downloadVideo = async (event) => {
    const videoId = event.target.id;
    const filename = event.target.title;
    console.log(filename);
    axios
      .get("http://localhost:3000/api/downloads/" + videoId + "/downloadfile", {
        responseType: "blob",
      })
      .then((response) => {
        FileDownload(response.data, `${filename}.mp4`);
      });
  };

  const removeVideo = async (event) => {
    const videoId = event.target.title;
    axios
      .delete("http://localhost:3000/api/downloads/" + videoId)
      .then((respsonse) => {
        window.location.reload();
      });
  };

  return (
    <div className="card" style={{ width: "18rem" }}>
      <img src={thumbnail} class="card-img-top" alt="thumbnail" />
      <div className="card-body">
        <h6 className="card-text">{title}</h6>
        <button
          id={_id}
          className="btn btn-success rounded"
          style={{ width: "100px" }}
          onClick={downloadVideo}
          title={title}
        >
          Download
        </button>
        <button
          title={_id}
          className="btn btn-danger rounded"
          onClick={removeVideo}
        >
          Delete
        </button>
      </div>
    </div>
  );
}
Enter fullscreen mode Exit fullscreen mode

Now let's run both frontend and backend code,

Backend will run on 3000 port => npm run dev

Frontend will run on 3001 port => npm start

Final YouTube Downloader

Source Code

Check out Upstash for production.

Top comments (7)

Collapse
 
iamkarshe profile image
Utkarsh Kumar Raut

npm install "thanks-for-share"

Collapse
 
kamalhossain profile image
Kamal Hossain

Hey there,
Thanks for sharing this.
I have two qustions, first ytdl-core has download speed issues, and sometimes the download stuck after downloading some part of it, do you have any solution plan for it?
Second, why did you created the the queue? Why is that important?
Thak you.

Collapse
 
pprathameshmore profile image
Prathamesh More
  1. No solution
  2. Queue will help to download files in the background. If server went down, and restarted it will all list of pending downloads in queue. Once server came back, it will start processing remaining downloads.
Collapse
 
kasiriveni profile image
Srinivas Kasiriveni

Nice👏😊😊👍

Collapse
 
iamjayant_verma profile image
Jayant Verma

Thanks for sharing

Collapse
 
abhishekbhard profile image
Abhishek

Thanks for sharing.

Collapse
 
msamgan profile image
Mohammed Samgan Khan • Edited

y dont you make your tool live for use.
its look like you got this done..
y just share the code, make a tool of it for use..

i can help you out, of needed in that......