Unlocking Creative Potential: Using LLMs for Music, Speech, and Art Generation

I built a Creative Copilot that turns a single text theme into three artifacts: a generated image, a spoken word audio track, and a melody written in ABC notation. It runs entirely on Oxlo.ai, which keeps costs predictable because you pay per request, not per token. This tutorial is for developers who want a concrete, multimodal pipeline without stitching together five different services.

What you'll need

• Python 3.10 or newer
• An Oxlo.ai API key from https://portal.oxlo.ai
• The OpenAI SDK installed with pip install openai

Step 1: Set up the client

I keep the whole pipeline on Oxlo.ai because the flat per-request pricing keeps the bill predictable even when the creative briefs get long. Start by importing the SDK and pointing it at the Oxlo.ai endpoint.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY"),
)

Step 2: Write the creative director prompt

The system prompt forces the model to return three tagged sections. I use Llama 3.3 70B as the general-purpose director because it follows formatting instructions reliably.

SYSTEM_PROMPT = """You are a Creative Director AI. When given a theme, generate three creative assets.

Output exactly in this format:

[IMAGE_PROMPT]
A detailed English prompt for a text-to-image model describing a single scene that matches the theme. Keep it under 100 words.

[SPOKEN_SCRIPT]
A 30-second spoken word script, poetic and rhythmic, inspired by the theme.

[ABC_MELODY]
A valid ABC notation tune with a title, key signature, meter, and melody that matches the mood of the theme. Include X:, T:, M:, K:, and the melody lines.

Do not write explanations outside the tags."""

Step 3: Generate a structured brief

This function sends the theme to the chat endpoint and parses the three tags into a dictionary.

def generate_brief(theme: str) -> dict:
    response = client.chat.completions.create(
        model="llama-3.3-70b",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Theme: {theme}"},
        ],
        temperature=0.9,
    )
    raw = response.choices[0].message.content

    def extract(tag: str, text: str) -> str:
        start = text.find(f"[{tag}]")
        if start == -1:
            return ""
        end = text.find("[", start + len(tag) + 2)
        if end == -1:
            end = len(text)
        return text[start + len(tag) + 2:end].strip()

    return {
        "image_prompt": extract("IMAGE_PROMPT", raw),
        "spoken_script": extract("SPOKEN_SCRIPT", raw),
        "abc_melody": extract("ABC_MELODY", raw),
    }

Step 4: Generate artwork

Pass the image prompt to the Oxlo.ai image generations endpoint. I use Flux.1 for high-quality illustration and save whatever the API returns to a local PNG.

def generate_art(prompt: str, path: str = "art.png"):
    res = client.images.generate(
        model="flux-1",
        prompt=prompt,
        size="1024x1024",
        n=1,
    )

    if res.data[0].url:
        import urllib.request
        urllib.request.urlretrieve(res.data[0].url, path)
    else:
        import base64
        from pathlib import Path
        img = base64.b64decode(res.data[0].b64_json)
        Path(path).write_bytes(img)

    print(f"Art saved to {path}")

Step 5: Synthesize speech

Oxlo.ai hosts Kokoro 82M for text-to-speech. Feed the spoken script into the audio endpoint and stream the result to an MP3.

def generate_speech(script: str, path: str = "speech.mp3"):
    res = client.audio.speech.create(
        model="kokoro-82m",
        voice="af",
        input=script,
    )
    res.stream_to_file(path)
    print(f"Speech saved to {path}")

Step 6: Export the ABC melody

Oxlo.ai does not need a specialized music model for this. The LLM already generated the melody as text, so we simply persist it to an ABC file that any standard notation tool can render.

def save_melody(abc: str, path: str = "melody.abc"):
    with open(path, "w", encoding="utf-8") as f:
        f.write(abc)
    print(f"Melody saved to {path}")

Step 7: Orchestrate the pipeline

Wire the steps together in a single script. Pass a theme as a command line argument, or fall back to a default.

if __name__ == "__main__":
    import sys

    theme = sys.argv[1] if len(sys.argv) > 1 else "a cyberpunk sunrise over a quiet ocean"

    print("Generating creative brief...")
    brief = generate_brief(theme)

    print("Generating artwork...")
    generate_art(brief["image_prompt"])

    print("Synthesizing speech...")
    generate_speech(brief["spoken_script"])

    print("Saving melody...")
    save_melody(brief["abc_melody"])

    print("Done. Check art.png, speech.mp3, and melody.abc.")

Run it

Save the script as creative_copilot.py, set your key, and run it.

export OXLO_API_KEY="sk-oxlo.ai-..."
pip install openai
python creative_copilot.py "neon gardens in the rain"

Expected output looks like this:

Generating creative brief...
Generating artwork...
Art saved to art.png
Synthesizing speech...
Speech saved to speech.mp3
Saving melody...
Melody saved to melody.abc
Done. Check art.png, speech.mp3, and melody.abc.

The ABC file will contain something usable like:

X:1
T:Neon Gardens
M:4/4
K:Emin
|:"Em"E2 G2 B2 e2|"C"c2 B2 G2 E2|"D"D2 F2 A2 d2|"Em"B2 A2 G2 F2:|

Next steps

Wrap this script in a FastAPI endpoint so users can POST a theme and receive the three files as a downloadable bundle. If you want longer, more elaborate creative reasoning, swap the director model to kimi-k2.6, or prototype for free by switching the chat calls to deepseek-v3.2 on the Oxlo.ai free tier.