We are building a symbolic music generator that turns text prompts into playable WAV files using an LLM and a lightweight numpy synthesizer. This tutorial is for developers who want to prototype generative audio tools without training custom models. We will optimize the LLM by constraining its output to a strict JSON schema and adding a self-correction loop for music theory compliance.
What you'll need
- Python 3.10 or newer
pip install openai numpy- An Oxlo.ai API key from https://portal.oxlo.ai. Oxlo.ai is a fully OpenAI-compatible platform, so the SDK works without changes.
Step 1: Define the system prompt
I start by locking the model into a strict JSON schema. The system prompt below embeds music theory constraints and a complete example so the LLM cannot hallucinate arbitrary pitch names or rhythmic values.
SYSTEM_PROMPT = """You are a symbolic music generator. Your output must be a single JSON object and nothing else.
Rules:
- The JSON must follow this schema:
{
"tempo": integer (BPM, 60-180),
"key": string (e.g., "C minor", "F# major"),
"time_signature": string (e.g., "4/4", "3/4"),
"tracks": [
{
"instrument": "piano",
"notes": [
{"pitch": string (e.g., "C4", "Eb5"), "start_beat": float, "duration_beats": float, "velocity": integer (1-127)}
]
}
]
}
- Every note pitch must belong to the declared key signature.
- Notes must align to a 1/16th beat grid (start_beat and duration_beats multiples of 0.25).
- The sum of duration_beats in each track must fill complete bars according to the time_signature.
- Do not write markdown, explanations, or code blocks. Output raw JSON only.
"""Step 2: Generate the first draft
Next I initialize the Oxlo.ai client and call Qwen 3 32B, which handles agentic instructions and structured formats well. I enable JSON mode so the response parses reliably into Python dictionaries.
import json
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
user_message = "A melancholic piano phrase in C minor, 4/4, 8 bars, moderate tempo"
response = client.chat.completions.create(
model="qwen-3-32b",
response_format={"type": "json_object"},
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
draft = json.loads(response.choices[0].message.content)
print(json.dumps(draft, indent=2))Step 3: Validate music theory
Raw LLM output often drifts outside the requested key or drops beats. I wrote a validator that checks every note against the declared key signature and measures whether the track fills complete bars.
NOTE_MAP = {
"C": 0, "C#": 1, "Db": 1, "D": 2, "D#": 3, "Eb": 3,
"E": 4, "F": 5, "F#": 6, "Gb": 6, "G": 7, "G#": 8,
"Ab": 8, "A": 9, "A#": 10, "Bb": 10, "B": 11,
}
SCALE_INTERVALS = {
"major": [0, 2, 4, 5, 7, 9, 11],
"minor": [0, 2, 3, 5, 7, 8, 10],
}
def pitch_to_midi(pitch: str) -> int:
octave = int(pitch[-1])
name = pitch[:-1]
return NOTE_MAP[name] + (octave + 1) * 12
def validate_track(track: dict, key: str, time_signature: str) -> list:
errors = []
root_name, mode = key.split()
root_pc = NOTE_MAP[root_name]
allowed = {(root_pc + i) % 12 for i in SCALE_INTERVALS[mode]}
beats_per_bar = int(time_signature.split("/")[0])
max_end = 0.0
for idx, note in enumerate(track.get("notes", [])):
pc = pitch_to_midi(note["pitch"]) % 12
if pc not in allowed:
errors.append(f"Note {idx} pitch {note['pitch']} is out of key {key}.")
if round(note["start_beat"] * 4) != note["start_beat"] * 4:
errors.append(f"Note {idx} start_beat {note['start_beat']} is not on 1/16 grid.")
if round(note["duration_beats"] * 4) != note["duration_beats"] * 4:
errors.append(f"Note {idx} duration {note['duration_beats']} is not on 1/16 grid.")
end = note["start_beat"] + note["duration_beats"]
max_end = max(max_end, end)
if max_end % beats_per_bar != 0:
errors.append(f"Track length {max_end} does not fill complete {time_signature} bars.")
return errors
errors = []
for trk in draft["tracks"]:
errors.extend(validate_track(trk, draft["key"], draft["time_signature"]))
print("Validation errors:", errors)Step 4: Self-correction loop
If the validator finds errors, I feed them back into the chat context and ask the model to regenerate the full JSON. This iterative refinement is the core optimization. It turns a flaky text generator into a reliable symbolic music engine.
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
{"role": "assistant", "content": json.dumps(draft)},
]
max_attempts = 3
for attempt in range(max_attempts):
if not errors:
break
fix_prompt = (
"The previous JSON has these validation errors:\n"
+ "\n".join(errors)
+ "\nRegenerate the entire JSON, fixing every error. Maintain the user's original intent."
)
messages.append({"role": "user", "content": fix_prompt})
response = client.chat.completions.create(
model="qwen-3-32b",
response_format={"type": "json_object"},
messages=messages,
)
draft = json.loads(response.choices[0].message.content)
messages.append({"role": "assistant", "content": json.dumps(draft)})
errors = []
for trk in draft["tracks"]:
errors.extend(validate_track(trk, draft["key"], draft["time_signature"]))
print(f"Attempt {attempt + 1} errors: {errors}")
final_track = draftStep 5: Synthesize to WAV
Once the JSON is clean, I render it to audio with a simple numpy synthesizer. Each note becomes a sine wave with an exponential decay envelope, mixed into a mono buffer and written to a standard WAV file.
import wave
import numpy as np
SAMPLE_RATE = 44100
def note_to_freq(pitch: str) -> float:
midi = pitch_to_midi(pitch)
return 440.0 * (2.0 ** ((midi - 69) / 12.0))
def render_track(track: dict, total_beats: float, bpm: int) -> np.ndarray:
seconds_per_beat = 60.0 / bpm
total_seconds = total_beats * seconds_per_beat
samples = int(np.ceil(total_seconds * SAMPLE_RATE))
buf = np.zeros(samples, dtype=np.float32)
for note in track["notes"]:
freq = note_to_freq(note["pitch"])
start_sec = note["start_beat"] * seconds_per_beat
dur_sec = note["duration_beats"] * seconds_per_beat
start_samp = int(start_sec * SAMPLE_RATE)
end_samp = min(int((start_sec + dur_sec) * SAMPLE_RATE), samples)
t = np.arange(end_samp - start_samp) / SAMPLE_RATE
envelope = np.exp(-t * 5.0)
wave = 0.2 * envelope * np.sin(2.0 * np.pi * freq * t)
buf[start_samp:end_samp] += wave
return buf
max_beat = 0.0
for trk in final_track["tracks"]:
for n in trk["notes"]:
max_beat = max(max_beat, n["start_beat"] + n["duration_beats"])
audio = render_track(final_track["tracks"][0], max_beat, final_track["tempo"])
audio = np.clip(audio, -1.0, 1.0)
pcm = (audio * 32767).astype(np.int16)
with wave.open("output.wav", "w") as w:
w.setnchannels(1)
w.setsampwidth(2)
w.setframerate(SAMPLE_RATE)
w.writeframes(pcm.tobytes())
print("Wrote output.wav")Run it
I save the complete script as music_gen.py, set my API key, and run it. The agent prints the validation attempts and writes the final audio file.
$ export OXLO_API_KEY="sk-oxlo.ai-..."
$ python music_gen.py
Validation errors: ['Note 3 pitch G#4 is out of key C minor.', 'Track length 34.0 does not fill complete 4/4 bars.']
Attempt 1 errors: []
Wrote output.wavPlaying output.wav gives a clean 8-bar piano phrase in C minor. The self-correction loop fixed the out-of-key accidental and padded the phrase to fill complete bars.
Wrap-up and next steps
This pipeline shows how an LLM can become a reliable music generation backend when you constrain its output and close the loop with validators. A concrete next step is to swap the sine-wave synth for a SoundFont renderer like fluidsynth for realistic instruments. Another is to cache successful prompt pairs and fine-tune the system prompt with few-shot examples of jazz or polyphonic voicings.
Because Oxlo.ai uses flat per-request pricing, running a three-turn correction loop on a long JSON prompt costs the same as a single ping. For production workloads, that predictability matters. See https://oxlo.ai/pricing for details.
Top comments (0)