The Role of LLMs in Arts and Design: A Technical Perspective

#engineering #oxlo #ai

We are building a Concept-to-Brief agent that turns a rough creative idea into a structured design document with image generation prompts, color rationale, and composition notes. It is for technical artists, creative coders, and design engineers who need reproducible, machine-readable briefs for rendering pipelines or design systems.

What you'll need

Python 3.10 or newer
An Oxlo.ai API key from https://portal.oxlo.ai
The OpenAI SDK: pip install openai

Step 1: Configure the Oxlo.ai client

I use llama-3.3-70b as the backbone because it handles long-form structured output reliably. On Oxlo.ai, flat per-request pricing means expanding a half-page concept into a full multi-section brief does not inflate cost as the context grows, which matters when you iterate on complex scenes. See https://oxlo.ai/pricing for plan details.

from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.oxlo.ai/v1",
    api_key="YOUR_OXLO_API_KEY"
)

MODEL = "llama-3.3-70b"

Step 2: Define the system prompt

The system prompt enforces a strict JSON schema so the agent emits machine-readable briefs we can feed directly into rendering tools or asset trackers.

SYSTEM_PROMPT = """You are a senior design technologist. Convert a rough creative concept into a structured brief.

Output strictly valid JSON with these keys:
- title: string
- summary: string, 1 to 2 sentences
- image_prompt: string, a detailed prompt suitable for an image generation model
- color_palette: array of objects, each with hex and rationale
- composition_notes: array of strings describing layout and visual hierarchy
- technical_constraints: array of strings, e.g., resolution, aspect ratio, render engine

Rules:
- Be specific. Reference concrete art movements, materials, or lighting models.
- Do not add markdown outside the JSON.
- Escape all special characters properly.
"""

Step 3: Generate the structured brief

This function sends the raw concept to the model and parses the returned JSON. I enable JSON mode to reduce formatting drift.

def generate_brief(concept: str) -> dict:
    response = client.chat.completions.create(
        model=MODEL,
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": concept},
        ],
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

Step 4: Validate style consistency

Creative pipelines often drift when multiple artists iterate on a concept. This validator compares a new variation idea against the approved brief and flags deviations.

VALIDATOR_PROMPT = """You are a design director reviewing a proposed variation against an approved brief.
Compare the variation to the brief and output strictly valid JSON with:
- approved: boolean
- deviations: array of strings describing mismatches
- suggestions: array of strings to align the variation with the brief
"""

def validate_variation(brief: dict, variation_idea: str) -> dict:
    user_msg = f"Approved brief:\n{json.dumps(brief, indent=2)}\n\nProposed variation:\n{variation_idea}"
    response = client.chat.completions.create(
        model=MODEL,
        response_format={"type": "json_object"},
        messages=[
            {"role": "system", "content": VALIDATOR_PROMPT},
            {"role": "user", "content": user_msg},
        ],
    )
    raw = response.choices[0].message.content
    return json.loads(raw)

Step 5: Assemble the pipeline

Wire both functions into a single script. The concept describes a retro-futuristic subway station, and the variation deliberately breaks style so we can see the validator catch it.

if __name__ == "__main__":
    concept = (
        "A retro-futuristic subway station rendered as a 3D matte painting, "
        "inspired by Syd Mead, with neon reflections on wet concrete, "
        "cinematic lighting, and a 21:9 aspect ratio."
    )

    print("=== Generating Brief ===")
    brief = generate_brief(concept)
    print(json.dumps(brief, indent=2))

    variation = (
        "The same station but hand-drawn in a watercolor sketch style "
        "with pastel colors and no neon lights."
    )

    print("\n=== Validating Variation ===")
    result = validate_variation(brief, variation)
    print(json.dumps(result, indent=2))

Run it

Save the file as design_agent.py, replace YOUR_OXLO_API_KEY with your key from https://portal.oxlo.ai, and run:

python design_agent.py

Typical output looks like this:

=== Generating Brief ===
{
  "title": "Neon Subway: Retro-Futuristic Transit Hub",
  "summary": "A cinematic 3D matte painting of a Syd Mead-inspired subway station emphasizing neon reflections and wet concrete surfaces.",
  "image_prompt": "Retro-futuristic subway station, 3D matte painting, Syd Mead aesthetic, neon tube lights casting cyan and magenta reflections on wet concrete floor, volumetric fog, cinematic lighting, wide 21:9 aspect ratio, high detail, octane render, 8k resolution",
  "color_palette": [
    {"hex": "#0A0A0F", "rationale": "Deep void black for ceiling and shadows"},
    {"hex": "#00F0FF", "rationale": "Cyan neon primary light source"},
    {"hex": "#FF0090", "rationale": "Magenta neon accent for depth"},
    {"hex": "#2A2A35", "rationale": "Cool concrete midtone"},
    {"hex": "#1C1C24", "rationale": "Wet floor reflection base"}
  ],
  "composition_notes": [
    "One-point perspective down the platform to draw the eye toward a vanishing point",
    "Neon tubes frame the top third of the image",
    "Reflections occupy the lower third to ground the scene"
  ],
  "technical_constraints": [
    "21:9 aspect ratio",
    "Minimum 4K resolution for print",
    "Octane or Unreal Engine 5 render",
    "HDR output for neon bloom accuracy"
  ]
}

=== Validating Variation ===
{
  "approved": false,
  "deviations": [
    "Medium changed from 3D matte painting to watercolor sketch",
    "Color palette shifted from neon cyan/magenta to pastel tones",
    "Removal of neon lights contradicts core lighting motif"
  ],
  "suggestions": [
    "Retain neon accents even in watercolor by using glowing edge washes",
    "Use pastel concrete but keep neon reflections in puddles",
    "Maintain 21:9 cinematic composition for brand consistency"
  ]
}

Next steps

Feed the generated image_prompt into Oxlo.ai's images/generations endpoint using Oxlo.ai Image Pro or Flux.1 to close the loop from text to pixel. If you want pixel-level critique, swap the validator model to kimi-k2.6 and pass the rendered image into a vision-enabled chat.completions call for direct visual feedback.

DEV Community