john jewski

Posted on Feb 22

Why I Stopped Writing Video Files to Disk and Route Everything Through /tmp

#webdev #ai #javascript #productivity

When I built the first version of dltkk.to I made the obvious choice — download the video file, save it to a downloads folder, stream it to the user, clean up on a timer.

Within a week I had three problems:

Disk filling up when cleanup timers misfired on server restarts
Stale files from failed downloads accumulating
Potential privacy issue if a user's file sat on disk longer than it should

Here's how I fixed all three by routing everything through /tmp.

The Original Architecture (Wrong)

// Old approach — save to disk, cleanup later
const outputDir = './downloads';
const filename = `video_${Date.now()}.mp4`;
const outputPath = path.join(outputDir, filename);

const ytdlp = spawn('yt-dlp', ['-o', outputPath, url]);

ytdlp.on('close', (code) => {
  res.download(outputPath, () => {
    fs.unlinkSync(outputPath); // What if this fails?
  });
});

// Cleanup timer as fallback
setInterval(() => {
  // Delete files older than 10 minutes
  // But what if server restarts?
}, 60000);

Problems:

If the server restarts mid-download the cleanup never runs
res.download callback doesn't guarantee the file was sent successfully
Disk space is finite and permanent storage compounds

The /tmp Architecture (Right)

function streamViaTmp(videoUrl, format, res) {
  const tmpFile = `/tmp/dltkk_${Date.now()}_${Math.random().toString(36).slice(7)}.mp4`;

  function cleanup() {
    try { if (fs.existsSync(tmpFile)) fs.unlinkSync(tmpFile); } catch(e) {}
  }

  const ytdlp = spawn('yt-dlp', ['-o', tmpFile, videoUrl]);

  ytdlp.on('close', (code) => {
    if (code !== 0 || !fs.existsSync(tmpFile)) {
      cleanup();
      return res.status(500).json({ error: 'Download failed' });
    }

    res.setHeader('Content-Disposition', 'attachment; filename="video.mp4"');
    const stream = fs.createReadStream(tmpFile);
    stream.pipe(res);
    stream.on('end', cleanup);      // Delete after streaming
    stream.on('error', cleanup);    // Delete on error too
  });

  // Kill and cleanup if client disconnects
  res.on('close', () => {
    if (!res.writableEnded) {
      ytdlp.kill('SIGTERM');
      cleanup();
    }
  });
}

Why this works:

/tmp is RAM-based on Linux — no actual disk writes
cleanup() is called in every code path — success, error, and disconnect
File only exists for the duration of the download + stream — seconds at most
Server restarts clear /tmp automatically on most Linux systems

The Stdout Trap

Before landing on /tmp I tried streaming yt-dlp output directly via stdout:

yt-dlp -o - URL | pipe_to_client

This looks elegant but breaks silently on any format requiring merging. YouTube downloads almost always need merging — separate video stream (f137) and audio stream (f140) combined by ffmpeg into a single MP4.

When you pipe to stdout, ffmpeg can't write the merged file atomically. It either fails silently or produces a corrupted file. You won't notice until a user complains their video has no audio.

The rule: anything that might need ffmpeg merging must go through a real file path, even if that file path is in RAM.

GIF Conversion — Two /tmp Files

GIF conversion needs two passes — download as MP4 first, then convert:

const tmpMp4 = `/tmp/dltkk_${ts}_${rand}.mp4`;
const tmpGif = `/tmp/dltkk_${ts}_${rand}.gif`;

// Download to tmpMp4 first
ytdlp.on('close', () => {
  exec(`ffmpeg -y -i "${tmpMp4}" -vf "fps=10,scale=480:-1:flags=lanczos" -loop 0 "${tmpGif}"`, () => {
    // Stream tmpGif to client
    const stream = fs.createReadStream(tmpGif);
    stream.pipe(res);
    stream.on('end', () => {
      fs.unlinkSync(tmpMp4); // Delete both
      fs.unlinkSync(tmpGif);
    });
  });
});

Both files stay in RAM, both get deleted after the GIF streams. At peak load you might have a few hundred MB in /tmp — fine for a VPS with 2GB+ RAM.

The res.on('close') vs req.on('close') Bug

One more trap worth mentioning. I had this:

req.on('close', () => {
  ytdlp.kill('SIGTERM'); // Kills yt-dlp immediately
  cleanup();
});

req fires close the moment the POST request body is fully read — which happens almost instantly after the request arrives. So I was spawning yt-dlp and killing it a fraction of a second later on every single download.

Fix: Use res.on('close') — that fires only when the actual browser connection drops.

res.on('close', () => {
  if (!res.writableEnded) {
    ytdlp.kill('SIGTERM');
    cleanup();
  }
});

What I Built With This

All of the above is running in production at dltkk.to — a free yt-dlp web frontend supporting 1000+ platforms.

DEV Community