AudioCraft Has a Free API — Meta's AI for Music, Sound Effects, and Audio

#ai #machinelearning #opensource #python

AudioCraft: Three AI Models in One Package

Meta's AudioCraft includes three powerful models:

MusicGen — generate music from text descriptions
AudioGen — generate sound effects from text
EnCodec — neural audio compression

All open-source, all running locally, all free.

The Free API (Python)

MusicGen — Text to Music

from audiocraft.models import MusicGen
from audiocraft.data.audio import audio_write

# Load model (small/medium/large/melody)
model = MusicGen.get_pretrained("facebook/musicgen-medium")
model.set_generation_params(duration=30)  # 30 seconds

# Generate from text
wav = model.generate([
    "epic orchestral battle music with drums and brass",
    "lo-fi hip hop beat with rain sounds"
])

for i, one_wav in enumerate(wav):
    audio_write(f"music_{i}", one_wav.cpu(), model.sample_rate)

AudioGen — Text to Sound Effects

from audiocraft.models import AudioGen

model = AudioGen.get_pretrained("facebook/audiogen-medium")
model.set_generation_params(duration=5)

wav = model.generate([
    "dog barking in a park",
    "thunder and heavy rain",
    "car engine starting"
])

Melody-Conditioned Generation

import torchaudio

model = MusicGen.get_pretrained("facebook/musicgen-melody")
model.set_generation_params(duration=30)

# Use an existing melody as reference
melody, sr = torchaudio.load("reference.wav")
wav = model.generate_with_chroma(
    ["jazz version of this melody"],
    melody[None].expand(1, -1, -1),
    sr
)

Model Comparison

Model	Parameters	VRAM	Quality	Speed
musicgen-small	300M	4GB	Good	Fast
musicgen-medium	1.5B	8GB	Better	Medium
musicgen-large	3.3B	16GB	Best	Slow
musicgen-melody	1.5B	8GB	Best+melody	Medium

Real-World Use Case

An indie game developer needed 100 unique background tracks. MusicGen generated them from descriptions like "peaceful forest ambient with birds" and "tense dungeon exploration music." 100 tracks in one afternoon, zero licensing fees.

Quick Start

pip install audiocraft
python -c "from audiocraft.models import MusicGen; m = MusicGen.get_pretrained(facebook/musicgen-small); print(Ready!)"

Resources

Need automated data collection for AI research? Check out my tools on Apify or email spinov001@gmail.com for custom AI pipelines.

DEV Community