Intro
Day 16!
Today's experiment: make a theme song for my family's cat — and see if it actually feels like their song, not a generic AI tune.
I used ACE-Step 1.5, a local music-generation AI (think "a Suno you can run at home"). It produced the whole package: lyrics, melody, and a singing voice. Everything here was generated locally on my own machine.
What I used: DGX Spark (my home AI machine) / ACE-Step 1.5 XL (music-generation AI).
Today's experiment
What I wanted to do
One theme song for my cat (Japanese vocals, ~3 minutes). Plus a few background tracks on the side, to measure how fast generation really is.
The approach
- First, the AI interviews me about my cat
- It turns my answers into lyrics (there's a "template" for which answer goes where)
- Hand the lyrics and a style prompt to ACE-Step 1.5 to generate the song
The goal
Not "huh, the AI made something," but a song I'd actually recognize as my cat's.
STEP 1: the AI interviewed me about my cat
Writing the lyrics started with me answering six questions.
| # | Question | My answer |
|---|---|---|
| 1 | What do you call them in the song? | "Our Nekko" (our little cat) |
| 2 | Personality in a word? | Super tsundere, super timid |
| 3 | Favorite gesture or habit? | Getting the base of their tail tapped |
| 4 | One memorable episode? | It takes at least six months to truly bond |
| 5 | How do you feel about them? | Calm — when they're nearby |
| 6 | Any song requests? | Bright and upbeat, female vocal |
"Tsundere" = prickly on the outside, secretly sweet on the inside. A very cat thing.
STEP 2: the template that turns answers into lyrics
Each answer lands in a fixed place in the song.
| Interview answer | Where it goes |
|---|---|
| The name | Chorus (repeated in the catchiest spot) |
| Personality | Verse 1 (introduce the character with everyday scenes) |
| Habit | Verse 2 (sharpen the picture with specifics) |
| Memory | Bridge (the emotional beat late in the song) |
| Feeling | The chorus's tone + the overall mood |
| Song request | Not lyrics — the style prompt |
With that template, my cat's chorus came out like this (Japanese, with a rough English gloss):
Uchi-no-Nekko wa tsun-tsun-tsundere (Our Nekko is tsun-tsun-tsundere)
Kokoro no tobira wa katai kedo (The door to their heart stays shut, but)
Kiiroi hitomi de chiratto ichibetsu (Those yellow eyes shoot one quick glance)
Dere wa kibun de nen ni suukai (The sweet side? A few times a year, if you're lucky)
STEP 3: generate. A 3-minute Japanese-vocal song
Hand ACE-Step 1.5 XL the lyrics and a style prompt (upbeat J-pop, female vocal, Japanese), and out came a full 3-minute song with Japanese singing.
Speed: the audio is 10 seconds, but "composing" takes 8 minutes
The interesting part was the time breakdown.
| What I made | Mode | Time |
|---|---|---|
| BGM (90s, no vocals) ×4 | quick | 40s total (~10s each) |
| Theme song (3min, vocals) | quick | ~10–15s each |
| Theme song (3min, vocals) | with composition planner | ~8 min (first run ~23 min) |
What's the "composition planner"? = the language model (LM) built into ACE-Step 1.5. It thinks through the song's blueprint (structure, metadata) before rendering audio. Better quality, but the thinking costs time.
Bonus: I made an MV too
How it's built: Whisper transcribes the lyric timings → an image AI (AnythingV5) makes a character and backgrounds → a video AI (LTX-2.3) animates the character → rembg cuts it out and composites it onto the background with subtitles (finer steps in the fold below).
Honestly, the video came out pretty cursed — the lower body suddenly morphs into a tail, the eyes go a little feral — so it's mildly unsettling. That's on the "improve later" list.
Today's takeaways
- The slow part isn't the sound, it's the thinking: the audio itself renders in ~10s; the long wait is the AI planning the song's structure
- Old tools converged: anime-style image (Day 11) + Whisper (Day 14) + video generation (Day 15) came together into a single MV
The details
:::details Environment and models
- Machine: DGX Spark (128GB unified memory)
- ACE-Step 1.5: set up from the official repo with
uv. DGX Spark (ARM64 + CUDA 13) is listed as an officially supported target, and it just ran - Model setup: DiT is
acestep-v15-xl-turbo(4B, 8 steps), planner LM is the 4B version. ~36GB of downloads total - Peak memory during generation ~27GB. Generated via the REST API server (easier to reproduce)
- MV side: ComfyUI (LTX-2.3 22B distilled fp8) + AnythingV5 (character & backgrounds) + rembg 2.0.76 (cutout, anime model isnet-anime) + Whisper large-v3 (timings) + ffmpeg (assembly) :::
:::details Full lyrics and the style prompt
Style prompt (Caption):
upbeat J-pop, energetic and heartwarming, female vocal, bright synth,
acoustic guitar, catchy chorus, 128 bpm, Japanese lyrics
Lyrics are passed with structural tags like [Verse] and [Chorus] (the words are Japanese; the （にゃー） bits are meow ad-libs sung as backing vocals):
[Intro]
（にゃー）
[Verse 1]
目が合った瞬間 ぷいっとそっぽ向く（にゃっ）
呼んでも来ないのに 気づけばそばにいる
物音ひとつで ロケットダッシュ（にゃー！）
ビビリなくせに 顔は堂々
[Pre-Chorus]
ツンとすまして 知らんぷり
それでもしっぽは 正直もの（にゃ？）
[Chorus - catchy]
うちのネッコは ツンツンツンデレ（にゃにゃ！）
心の扉は かたいけど
黄色い瞳で ちらっと一瞥
デレは気分で 年に数回（にゃー）
[Verse 2]
撫でられるのは 好きじゃないくせに
しっぽの付け根を とんとん叩けば
目を細めて 喉を鳴らして
もっと続けてと 視線で命令（にゃっ）
[Bridge - emotional]
心が通じるまで 半年かかった
ゆっくりゆっくり 縮めた距離は
いまでは世界で いちばん近い
気づけば隣が 定位置になった
[Chorus - anthemic]
うちのネッコは ツンツンツンデレ（にゃにゃ！）
ビビリなところも ご愛嬌
とんとんのリズムで しっぽが揺れる
今日も我が家の 王様です（にゃー！）
[Outro - fade out]
うちのネッコ（にゃー）
うちのネッコ（にゃー にゃー）
A tip from the official docs: put the fine-grained style control in the Caption, and keep the lyric tags simple.
:::
:::details How the MV was built (the finer steps)
- Uses the intro through the second chorus (~70s). Per-line timings from Whisper large-v3 (since I already know the lyrics, a mis-hear here and there is fine — I only use the timestamps)
- 20+ cuts. One lyric line = one cut, each assigned a clip whose motion fits the words
- The character image is a single frame: my "anime-fied cat" from Day 11, redrawn into a mascot with an image AI (AnythingV5). That's handed to the video AI (LTX-2.3) to animate. ~30s per clip; motion is described in plain requests ("turn away," "walk cycle," etc.)
- Cutout compositing is frame-by-frame: rembg (isnet-anime) cuts out the character → composite onto a slowly zooming/panning background → draw subtitles → re-encode at 24fps
- Keeping the character's scale fixed within a cut mattered. Matching to the per-frame silhouette makes it shrink the instant it stretches (learned the hard way) :::
:::details Gotchas
- The dash comes last: when I asked LTX-2.3 to "suddenly bolt away," the motion tended to land in the clip's final 0.5s. Worked around it by shifting the segment I use
- Feed the input image in the aspect ratio you want out: at first I fed a portrait (512×768) character image into a landscape (768×512) generation, and every clip came out as an upper-body zoom that cut off the ears. Rebuilt the input as a landscape full-body image to fix it. Video AIs inherit the input image's composition quite strongly
-
The planner may have been running slower than it should: digging through the logs, the LM's fast-execution component (vLLM) seems to have failed to compile due to a missing
Python.h, so it may have quietly fallen back to a slow path (I didn't fully chase down the cause on Day 16). I hit the same root cause on Day 15 (python3.12-devnot installed) — could be a recurring Spark gotcha :::
:::details License notes
- ACE-Step 1.5: code and weights both MIT. Commercial use OK
- Planner LM (4B, Qwen3-4B based): also MIT
- The generated music is explicitly cleared for commercial use in the official docs (training data is licensed + royalty-free + synthetic)
- MV-side tools (LTX-2.3, AnythingV5, rembg, Whisper) are the ones I already vetted on Days 11–15 :::
Sources
Outro
Lyrics, melody, singing, and a music video — all AI, all on my own machine. See you next time!
Thanks for reading!
Top comments (0)