Music Generation with LLM: Opportunities and Challenges

We are going to build a command-line melody generator that turns a text description into a playable WAV file. It uses an LLM to compose a structured score, then synthesizes the audio locally with Python. This is useful for developers prototyping generative audio workflows or adding dynamic soundtracks to apps and games.

What you'll need

Python 3.10 or newer installed locally. The openai Python SDK and numpy:

pip install openai numpy

An Oxlo.ai API key from https://portal.oxlo.ai. Oxlo.ai uses flat per-request pricing, so experimenting with long, detailed composition prompts does not inflate your cost the way token-based billing would. You can view the details at https://oxlo.ai/pricing.

Step 1: Set up the Oxlo.ai client

I start by initializing the OpenAI-compatible client pointing at Oxlo.ai. I will use deepseek-v3.2 because it handles structured output and coding tasks reliably, and it is available on the free tier.

from openai import OpenAI

client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")

Step 2: Write the system prompt

The system prompt locks the model into a strict JSON schema so we can parse the melody automatically. I keep the instructions unambiguous and forbid markdown wrapping.

SYSTEM_PROMPT = """You are a music composition engine.
Given a user description, generate a short melody as a JSON object.
The JSON must contain exactly two keys:
- "tempo": an integer BPM between 60 and 180
- "notes": an array of objects, each with "frequency" (Hz, float) and "duration" (seconds, float)
Select frequencies that correspond to real musical pitches. Keep total duration under 10 seconds.
Output ONLY raw JSON. Do not wrap the output in markdown code fences or add commentary."""

Step 3: Generate a structured score

This function sends the user's description to Oxlo.ai and parses the returned JSON. I use deepseek-v3.2 for reliable structured reasoning.

import json

def compose_melody(description: str):
    response = client.chat.completions.create(
        model="deepseek-v3.2",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": description},
        ],
    )
    raw = response.choices[0].message.content.strip()
    # Strip accidental markdown fences
    if raw.startswith("

```"):
        raw = raw.split("\n", 1)[1].rsplit("```

", 1)[0].strip()
    return json.loads(raw)

Step 4: Synthesize the audio

I synthesize the waveform with pure sine waves. No external audio libraries are required beyond numpy.

import numpy as np
import wave
import struct

SAMPLE_RATE = 44100

def generate_wav(score: dict, output_path: str = "output.wav"):
    tempo = score["tempo"]
    notes = score["notes"]
    samples = []

    for note in notes:
        freq = float(note["frequency"])
        duration = float(note["duration"])
        t = np.linspace(0, duration, int(SAMPLE_RATE * duration), endpoint=False)
        wave_data = 0.5 * np.sin(2 * np.pi * freq * t)
        # Simple attack/decay envelope to avoid clicking
        envelope = np.minimum(np.linspace(1, 0, len(t)), np.linspace(0, 1, len(t)))
        samples.extend(wave_data * envelope)

    audio = np.array(samples)
    audio = (audio * 32767).astype(np.int16)

    with wave.open(output_path, "w") as f:
        f.setnchannels(1)
        f.setsampwidth(2)
        f.setframerate(SAMPLE_RATE)
        f.writeframes(audio.tobytes())

    return output_path

Step 5: Wrap it in a CLI

The final script ties the composition and synthesis together. Running it reads a prompt from arguments and writes output.wav.

import sys

if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python music_gen.py 'a cheerful morning melody in C major'")
        sys.exit(1)

    user_prompt = sys.argv[1]
    print(f"Composing: {user_prompt}")
    score = compose_melody(user_prompt)
    print(f"Generated score at {score['tempo']} BPM with {len(score['notes'])} notes")
    path = generate_wav(score)
    print(f"Wrote audio to {path}")

Run it

Execute the script with a descriptive prompt. Here is what a typical session looks like:

$ python music_gen.py "a slow ambient drone in D minor"
Composing: a slow ambient drone in D minor
Generated score at 72 BPM with 8 notes
Wrote audio to output.wav

You can open output.wav in any audio player. If the model returns malformed JSON, the script will raise a json.JSONDecodeError. In production, you would add a retry loop or fall back to a constrained mode like JSON mode if your model supports it.

Next steps

Swap deepseek-v3.2 for qwen-3-32b or llama-3.3-70b on Oxlo.ai to compare how different architectures handle rhythmic structure in the JSON output. For polyphonic arrangements, extend the schema so each note carries a list of simultaneous frequencies and mix the sine waves per time slice.