DEV Community

Sane Devs
Sane Devs

Posted on

Wrapping Demucs in FastAPI: Building a Music Stem Separation Web App

Audio stem separation has improved dramatically over the past few years, largely driven by open-source deep learning architectures like Demucs from Meta AI. If you want to isolate clean vocals from a track or strip them away to create an instrumental backing, the results from models like htdemucs (Hybrid Transformer Demucs) are genuinely impressive. However, running these tools out of the box is strictly a terminal-bound workflow. You have to clone a repository, set up a Python virtual environment, configure PyTorch and CUDA drivers, and run terminal commands with custom parameter flags.

For non-technical musicians, producers, or colleagues who just need an isolated vocal track for a demo, the terminal is an unnecessary barrier. At the same time, commercial online stem splitters often gate their services behind steep subscription paywalls, intrusive advertisements, or strict file size limits.

As a two-person team, we built MusicSpliter as a lightweight prototype to explore how to wrap an open-source deep learning CLI tool into a straightforward web application. The concept is simple: drop an MP3 into a browser window, click a button, let the model process the audio on the backend, and download the isolated vocal and accompaniment stems as WAV files.

Here is how we structured the application between FastAPI and React, how we handled the CLI process execution, and the practical challenges you run into when serving heavy machine learning workloads over standard HTTP endpoints.

The Architecture

When building a prototype around a deep learning CLI tool, simplicity helps you validate the core experience quickly. We chose a two-tier architecture:

  1. Frontend (React, TypeScript, Vite): A minimal single-page interface that accepts an MP3 file, issues a multipart form upload to the API, indicates an active processing state, and surfaces download buttons once the output stems are ready.
  2. Backend (FastAPI, Python, Demucs): A lightweight API service that receives the uploaded audio file, saves it to a designated workspace directory, invokes the Demucs CLI via Python's subprocess module, and serves the generated audio files back to the client using static file mounting.

Instead of writing custom audio streaming endpoints or dealing with temporary memory buffers, we leveraged Demucs' predictable directory conventions. When Demucs processes an audio track, it writes its output to a structured directory hierarchy based on the model name and track title. By mounting that output folder as a static directory in FastAPI, the frontend can download the processed stems directly via URL.

Let us walk through how this is implemented step by step.

Step 1: Setting Up the Backend and Static File Serving

In backend/main.py, we initialize FastAPI, enable Cross-Origin Resource Sharing (CORS) so our Vite development server can communicate with the backend, and set up our path variables.

from fastapi import FastAPI, UploadFile, File
from fastapi.responses import JSONResponse
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
import subprocess
import os

app = FastAPI()

app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:5173"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
OUTPUT_FOLDER = os.path.join(BASE_DIR, "demucs_output")
MODEL = "htdemucs"

os.makedirs(OUTPUT_FOLDER, exist_ok=True)

# Serve demucs_output as static files
app.mount("/files", StaticFiles(directory=OUTPUT_FOLDER), name="files")
Enter fullscreen mode Exit fullscreen mode

Setting up OUTPUT_FOLDER at the project root ensures all processed files live in a single predictable location. Mounting StaticFiles at /files allows any file written inside demucs_output to be downloaded directly by the browser without writing dedicated file-streaming routes.

Step 2: Ingesting the Uploaded Audio File

Next, in backend/main.py, we define our /split endpoint. When a user uploads a track, FastAPI receives it as an UploadFile object.

@app.post("/split")
async def split_song(file: UploadFile = File(...)):
    # Save uploaded file
    input_path = os.path.join(BASE_DIR, file.filename)
    with open(input_path, "wb") as f:
        f.write(await file.read())
Enter fullscreen mode Exit fullscreen mode

FastAPI handles multipart form uploads efficiently. By reading the file asynchronously and writing it to disk at input_path, we ensure the file is fully persisted before we invoke the Demucs command-line tool.

Step 3: Invoking Demucs and Building Stem URLs

Once the MP3 is saved to disk, we trigger the separation process using Python's subprocess.run. We specify the model, the output directory, and the separation target.

    # Run demucs
    try:
        subprocess.run([
            "demucs",
            input_path,
            "-o", OUTPUT_FOLDER,
            "-n", MODEL,
            "--two-stems=vocals"
        ], check=True)
    except subprocess.CalledProcessError as e:
        return JSONResponse(status_code=500, content={"error": str(e)})

    song_name = os.path.splitext(os.path.basename(input_path))[0]
    vocals_url = f"/files/{MODEL}/{song_name}/vocals.wav"
    karaoke_url = f"/files/{MODEL}/{song_name}/no_vocals.wav"

    return {"vocals": vocals_url, "karaoke": karaoke_url}
Enter fullscreen mode Exit fullscreen mode

A few important details are happening here:

  • The --two-stems=vocals flag: By default, Demucs splits audio into four separate stems: drums, bass, vocals, and other. For vocal removal or karaoke creation, separating four stems takes twice as long and requires significantly more GPU memory. The --two-stems=vocals flag instructs the model to only isolate the vocal track and combine everything else into a single accompaniment stem.
  • Output file conventions: When using the htdemucs model with --two-stems=vocals, Demucs creates a subfolder named after the input file without its extension. Inside that subfolder, it outputs two files: vocals.wav and no_vocals.wav.
  • URL generation: Because we mounted OUTPUT_FOLDER to /files, we construct the client-accessible URLs directly using the model name and track name, returning them in a JSON payload.

Step 4: Handling the Frontend Workflow and Upload State

On the frontend, we built a React component in frontend/src/App.tsx that manages the user interaction: file selection, uploading, progress indication, and downloading the resulting files.

import React, { useState } from "react";

const API_URL = "http://127.0.0.1:8000/split";

function App() {
  const [file, setFile] = useState<File | null>(null);
  const [loading, setLoading] = useState(false);
  const [result, setResult] = useState<{ vocals: string; karaoke: string } | null>(null);

  const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
    setFile(e.target.files?.[0] || null);
    setResult(null);
  };

  const handleUpload = async () => {
    if (!file) return;
    setLoading(true);
    setResult(null);

    const formData = new FormData();
    formData.append("file", file);

    try {
      const res = await fetch(API_URL, {
        method: "POST",
        body: formData,
      });
      const data = await res.json();
      setResult(data);
    } catch (err) {
      alert("Error splitting music!");
    }
    setLoading(false);
  };

  return (
    <div style={{ maxWidth: 400, margin: "40px auto", padding: 20, border: "1px solid #ccc", borderRadius: 8 }}>
      <h2>Music Spliter</h2>
      <input type="file" accept=".mp3" onChange={handleFileChange} />
      <br /><br />
      <button onClick={handleUpload} disabled={!file || loading}>
        {loading ? "Splitting..." : "Split Music"}
      </button>
      {loading && (
        <div style={{ margin: "20px 0" }}>
          <div style={{ width: "100%", background: "#eee", height: 10, borderRadius: 5 }}>
            <div style={{ width: "80%", background: "#007bff", height: 10, borderRadius: 5, transition: "width 0.5s" }} />
          </div>
          <p>Processing...</p>
        </div>
      )}
      {result && (
        <div style={{ marginTop: 30 }}>
          <h4>Download Options:</h4>
          <div>
            <a href={`http://127.0.0.1:8000${result.vocals}`} download>
              <button>Download Vocals</button>
            </a>
          </div>
          <div style={{ marginTop: 10 }}>
            <a href={`http://127.0.0.1:8000${result.karaoke}`} download>
              <button>Download Karaoke</button>
            </a>
          </div>
        </div>
      )}
    </div>
  );
}

export default App;
Enter fullscreen mode Exit fullscreen mode

Because audio stem separation takes anywhere from 15 seconds to over a minute depending on track length and hardware, the user interface disables the submission button immediately on click and displays an active progress indicator. When the response arrives, the user receives separate download buttons for the vocal track and the karaoke backing track.

Gotchas and Real-World Edge Cases

While wrapping a CLI tool in FastAPI works quickly for a local proof-of-concept, taking this pattern further reveals several operational gotchas you have to address in production.

1. Blocking the Python Event Loop

In backend/main.py, subprocess.run is a synchronous, blocking call executed directly inside an async def route. In Python's asynchronous event model, if a synchronous function runs for 45 seconds inside the event loop, the entire FastAPI server is blocked from processing any other incoming HTTP requests during that time.

For a local single-user prototype, this goes unnoticed. But in any shared environment, you should offload the subprocess execution to a worker thread using asyncio.to_thread or move the task entirely to a background worker queue.

2. Error Trapping and Missing System Dependencies

Demucs depends heavily on external libraries like FFmpeg to decode MP3 files into raw audio buffers. If FFmpeg is not installed on the server, or if the uploaded audio file has an uncommon sample rate or corrupted ID3 tags, Demucs exits with a non-zero status code.

Catching subprocess.CalledProcessError prevents the API process from crashing, but returning just str(e) often omits the actual cause. Capturing stderr (via capture_output=True) is essential to see whether Demucs failed due to a missing binary, an unsupported codec, or a CUDA out-of-memory error.

3. Filename Sanitization and Overwrites

In our prototype code, we used file.filename directly:

input_path = os.path.join(BASE_DIR, file.filename)
Enter fullscreen mode Exit fullscreen mode

In production, relying on raw client filenames introduces two significant risks:

  1. Path traversal and unusual characters: Filenames containing spaces, quotes, or directory traversal characters (../) can break shell arguments or create unexpected paths.
  2. Name collisions: If two different users upload a file named track.mp3, the second upload will overwrite the first input file and overwrite the files in the output directory.

A safer pattern is to generate a unique identifier (such as a UUID) for each job, store the file under that ID, and maintain the original filename only as metadata for the eventual download header.

4. Disk Consumption from Uncompressed WAV Files

Demucs outputs uncompressed WAV audio files (often 24-bit or 32-bit float). A standard 4-minute MP3 that starts at 8 MB can easily expand into two 40 MB WAV files once separated.

If you process twenty songs, your server quickly accumulates gigabytes of static audio files in demucs_output. Without an automated cleanup routine (such as a cron job or background task that deletes folders older than 60 minutes), local storage will fill up fast.

5. HTTP Timeout Limits

Because machine learning inference is compute-intensive, processing longer audio files can easily exceed standard HTTP timeout thresholds. If your backend sits behind a reverse proxy like Nginx or a cloud proxy like Cloudflare, any request taking longer than 60 to 100 seconds will be terminated with a 504 Gateway Timeout, even if Demucs is still happily crunching away on the GPU.

What We Would Do Differently: Architecture Trade-Offs

Reflecting on the prototype, here is how we would evolve the architecture for a multi-user environment:

  • Asynchronous Task Queue: Instead of holding an open HTTP connection while the model runs, we would use an asynchronous worker system like Celery or Redis Queue (RQ). The /split endpoint would immediately return a job_id, and a dedicated worker process would handle the Demucs execution.
  • Polling or WebSockets for Progress: Rather than an animated placeholder progress bar, the client could poll a /status/{job_id} endpoint or listen over a WebSocket. Demucs outputs progress percentage to stdout, which can be parsed and streamed back to the frontend in real time.
  • In-Browser Audio Preview: Instead of only providing direct download links, embedding an HTML5 <audio> player or waveform visualizer would let users audition the separated vocals and instrumental tracks immediately before deciding to download.
  • Object Storage Lifecycle: Rather than storing audio files on the local application disk, we would upload the generated stems to an S3-compatible bucket (like Cloudflare R2 or AWS S3) with signed temporary download URLs and an automated 24-hour expiration rule.

Summary

  • Wrapping a machine learning CLI tool with FastAPI and static file mounting is an effective way to prototype an AI web application without rewriting low-level inference code.
  • Using Demucs' --two-stems=vocals flag dramatically cuts processing time and VRAM usage when you only need vocal and instrumental separation.
  • Synchronous calls like subprocess.run block FastAPI's async event loop unless offloaded to a thread pool or an external queue.
  • Real-world deployments require robust filename sanitization, disk retention policies, and asynchronous job tracking to avoid reverse-proxy timeouts.

Have you wrapped deep learning or audio CLI tools in web services before? How do you prefer to handle long-running ML tasks and progress feedback between FastAPI and your frontend?

The full write-up of this project is on our site: https://sanedevs.com/projects/musicspliter.html

Top comments (0)