Zero cost. Zero external calls. On a single M1 Max, two 100-second vocal songs (two seed variants) come out in about 58 seconds (measured on this M1 Max 64GB setup — details below).
The audience I have in mind: Apple Silicon folks running LLMs/image/video models locally, and musicians who want to avoid cloud billing. "Suno is convenient, but I hate the billing and the cloud upload. Does vocal-song generation really work on just a Mac, and how fast is it actually?" — this post answers that with real measurements.
This is the second entry in my "free local AI ___, the honest log" series. Last time it was video → I Built a 100% Free, Fully Local AI Short-Video Pipeline on an M1 Mac. This time it's music. It's the real log of producing eight English learning songs for kids — and I'll start with where I got burned (the failures).
TL;DR
-
To verify "it's running locally," don't trust a passing health check — inspect the endpoint URL and
api_mode. The CLI bundled in the repo points to the cloud (acemusic.ai) by default, and the health check keeps returning OK even while your local server sits there perfectly healthy (the biggest trap; details below). - Measured on an M1 Max 64GB: two variants of a 100-second vocal song = 58.48s (LM 15.70s + DiT 42.78s). Total generation for all 8 songs × 2 variants: about 9 minutes.
- Don't judge quality by ear alone: a 3-layer scoring stack — mlx-whisper transcription match rate × an independent rubric × your ears. The final 8 keepers averaged an 83.9% match rate and 94.1 points on the independent rubric.
- The environment landmine is the same one as last time (the video post): arm64 Python. This time it was uvx grabbing an anaconda-flavored x86 Python.
1. The incident: "I thought it was local — it was the cloud." Never trust a passing health check
Zero-cost, zero-external-calls AI music — that was the plan. In reality, my generation requests were flying off to the cloud. It's the most valuable failure in this log, so it goes first.
Timeline
- Day one.
git clone+uv sync, API server up onlocalhost:8001. Smooth so far. - The repo ships a CLI (
acestep.sh) as.claude/skills, and as I started generating with it — - The CLI's default endpoint was the cloud API (acemusic.ai). Even with a perfectly healthy API server running on my machine, requests were configured to go external.
Why you won't notice: the cloud health check returns OK
Here's the nasty part. The CLI's health check returns "OK" — as a health check of the cloud side. Your local server also started fine. So "server up OK, health check OK, generation works" — from every angle it looks like "it works," while the reality is external transmission. The more strictly you require fully-local (zero external calls), the more this cosmetic OK will fool you.
The fix is two lines
Point the endpoint and the API mode explicitly at local:
bash scripts/acestep.sh config --set api_url "http://127.0.0.1:8001"
bash scripts/acestep.sh config --set api_mode native
Lesson: If your requirement is fully local (zero external calls, commercial-safe), don't accept "health check OK" — inspect the endpoint URL and
api_modeyourself. A passing health check and requests staying local are two different questions.
Every measurement below was taken after this fix (api_mode native, targeting 127.0.0.1).
2. Why ACE-Step 1.5 — model selection from a Mac perspective
I compared the usual "local AI music on a Mac" candidates along three axes: vocals (songs with singing), runs on a Mac, and license. Every license claim below is primary-sourced from the official repo/model card (as of July 2026).
| Model | Why I passed | License (primary-sourced) |
|---|---|---|
| MusicGen / AudioCraft | Weights are CC-BY-NC 4.0 = no commercial use (code is MIT). Also, vocals were removed from the training data via source separation, and the official card states "The model is not able to generate realistic vocals." = can't make songs | Code=MIT / weights=CC-BY-NC 4.0, model card |
| YuE | No Mac path. The official repo states "FlashAttention 2 is mandatory" = CUDA-first, with no macOS/Apple Silicon support mentioned. (License itself is Apache 2.0, which is fine) | Apache 2.0 |
| Stable Audio Open | The official model card states "The model is not able to generate realistic vocals." = no vocal songs | Stability AI Community License |
| DiffRhythm | Code and DiT weights are Apache 2.0, but the VAE alone is under the Stability AI Community License (officially: "DiffRhythm-VAE is subject to the Stability AI Community License Agreement") — licensing splits per component. Messy to manage | Code/DiT=Apache 2.0, VAE=Stability AI Community License |
By the way, MusicGen — which I used for BGM in the previous post — is, per the table, "non-commercial + can't sing," so it stopped at BGM. Needing a different model for actual songs is where this project started.
What sold me on ACE-Step 1.5 — three things:
- MIT license (no strings for commercial use; compared with the table above, that simplicity is precious)
-
Official macOS/Apple Silicon support. A
start_api_server_macos.shships in the repo; the LM side runs on MLX and the DiT side on PyTorch MPS - A bundled REST API server, easy to wire into batch production and automation
The LM comes in 0.6B/1.7B/4B. I started with 0.6B. There's an external report of 1.7B peaking at 42GB (not verified by me), so on a machine that also keeps other local LLMs resident, skipping it felt like the safe call.
3. Install and launch (M1 Max 64GB, ~7GB initial download)
The install itself is straightforward, so I'll keep this short.
Environment: MacBook Pro M1 Max 64GB (unified memory). On startup the server detected "51.8GB unified memory, tier=unlimited". Models: LM = acestep-5Hz-lm-0.6B + DiT = acestep-v15-turbo.
# Start the API server (port 8001, LM on the MLX backend)
ACESTEP_LM_BACKEND=mlx TOKENIZERS_PARALLELISM=false \
uv run acestep-api --host 127.0.0.1 --port 8001 --lm-model-path acestep-5Hz-lm-0.6B
# Confirm it's listening
lsof -nP -iTCP:8001 -sTCP:LISTEN
The first run downloads the models. Measured: about 7GB total under checkpoints/ (about 4 minutes on a 47MB/s connection).
Three small traps:
-
start_api_server_macos.shcan pop an interactive prompt during its git update check. Callinguv run acestep-apidirectly is the reliable path. - The bundled CLI
scripts/acestep.shdoesn't have its executable bit set. Call it viabash. - The initial download and long generations will time out if you run them synchronously. Run them in the background.
4. Measured: two 100-second vocal songs in 58.48s (LM 15.70s + DiT 42.78s)
The main event. Generation is just handing the CLI a caption (style tags), lyrics with structure tags, and parameters.
bash scripts/acestep.sh generate -c "<caption>" -l "<full lyrics with structure tags>" \
--duration 100 --bpm 100 --key-scale "C major" --language en
Here's the result metadata for song 1 (a 100-second ABC song). One request auto-generates two variants with different seeds, and the recorded time is the total for both.
"generation_info": "**🎵 Total generation time (2 songs): 58.48s**
- 29.24s per song
- LM phase (2 songs): 15.70s
- DiT phase (2 songs): 42.78s",
"seed_value": "1014051477,117013642"
Two finished 100-second songs in 58.48s. Generation runs faster than the song's actual length — faster than real time. The breakdown: LM (lyrics → semantic tokens) 15.70s, DiT (audio synthesis) 42.78s. Having this breakdown tells you which side to lighten if you want more speed, so I recommend keeping every meta.json.
Measurements across all 8 songs
That pace wasn't first-song luck; it held across all eight.
| Song | Length | Generation time (2 variants) |
|---|---|---|
| 01 | 100s | 58.48s |
| 02 | 110s | 66.32s |
| 03 | — | 62.99s |
| 04 | — | 63.97s |
| 05 | — | 68.92s |
| 06 | — | 65.55s |
| 07 | 120s | 74.91s |
| 08 | 115s | 74.90s |
Total generation time for all 8 songs × 2 variants (16 tracks): about 9 minutes. The outputs are 100–125s mp3s at 48kHz/128kbps, roughly 1.6MB (100s) to 2.0MB (125s) each. The lyrics were synthesized word-for-word as written (verified by inspecting metas.lyrics in meta.json).
All numbers are measured on this M1 Max 64GB configuration. I haven't measured M2/M3/M4, so I'm not extrapolating to other chips. This is not a promise of "58 seconds for everyone."
5. Lyric design: the LMNOP problem is universal
Fast generation is useless if the lyric design is off. From here on, this is operational field experience you won't find in the usual roundups.
The "elemenopee" problem
In the traditional ABC-song melody, L-M-N-O-P gets crammed together and smears into "elemenopee" — a well-known issue. I avoided it by regrouping the letters into evenly spaced chunks: ABCD / EFG / HIJK / LMN / OPQ / RST / UVW / XYZ.
This isn't just about the alphabet — it's a principle for any enumeration lyric (numbers, days of the week): place each item at even intervals and don't insert "and" into the enumeration ("twenty-nine and thirty" was actually flagged in scoring).
General techniques for clearer AI singing
Lyric-side techniques that solidified in practice, and should apply beyond ACE-Step:
- Write numbers as English words ("30" → "thirty"; the transcription scoring needed number normalization too)
- Have letters sung one at a time with separators:
A - B - C - 4–8 words and 6–10 syllables per line; keep parallel lines within ±1–2 syllables
- Repeat choruses with identical spelling, word for word. Use parentheses for backing vocals
(happy!)and structure tags like[call and response] - Keep the caption to 4–7 consistent tags, and never let caption and lyric moods contradict. Don't put BPM/key in the caption — pass them as dedicated parameters
- Three prohibitions that came out of independent scoring: 1) no "and" inside enumerations; 2) don't contort English for the sake of rhyme ("but fun, all right!" was a real example); 3) at most one alteration per chorus, and keep the same enumeration consistent within a song (no mixing fall vs autumn)
The real thing: song 1's full lyrics and parameters
- Parameters: duration 100s / BPM 100 / C major / vocal en
- caption:
children's song, cheerful, playful, simple melody, ukulele, glockenspiel, acoustic guitar, clear female vocals, warm, sing-along
[Intro]
[Verse]
A - B - C - D
E - F - G
H - I - J - K
L - M - N
[Verse]
O - P - Q
R - S - T
U - V - W
X - Y - Z
[Chorus]
Now I know my A - B - C (A - B - C!)
Sing the alphabet with me (sing with me!)
Happy letters, you and me
Come and sing the A - B - C
[Verse] (repeat)
[Chorus] (repeat)
[Outro]
A - B - C, you and me!
These lyrics, as-is, became the two variants in the 58.48s above (measured 48kHz, 100.03s each).
6. "Don't grade your own songs" — the 3-layer Whisper scoring stack
Comparing 16 tracks entirely by ear to rank them is, honestly, brutal. So I inserted objective scoring via transcription — and stepped on another landmine setting it up, so let's start there.
Failure #2: uvx grabs an x86 Python
I tried to install mlx-whisper (fully local transcription) for scoring via uvx, and dependency resolution failed. The cause: this machine's default python3 is an anaconda-provided x86_64 (Rosetta) build, and uvx grabbed it. mlx is arm64-only, so it won't install.
It's the sequel to the "make it grab arm64 Python" landmine from the previous post. You can avoid it when you create venvs yourself — but it comes back through convenience wrappers like uvx. Passing an arm64 CPython explicitly via --python gets you through:
uvx --python cpython-3.12-macos-aarch64-none --from mlx-whisper \
mlx_whisper <mp3> --model mlx-community/whisper-large-v3-turbo
When an MLX tool fails mysteriously on Apple Silicon, first check your arch with python3 -c "import platform; print(platform.machine())" — especially on machines that also have anaconda.
The three layers
- Objective layer: transcribe each track with mlx-whisper (large-v3-turbo, Apache 2.0, fully local) and compute the word-sequence match rate against the lyrics actually used for synthesis (verified from meta.json). Normalize numbers to English words before comparing (to correct false-low scores from "1,2,3"-style transcriptions)
- Independent layer: have an agent in a separate context score against a rubric (English correctness 25 / design-principle adherence 25 / educational value 25 / singability 25 = 100 points). If the author (the same-context session) grades it, the grades go soft — so separate them
- Final layer: human ears. A person covers Whisper's measurement limits (below)
What the data actually said
Match rates spread far more than I expected. Across the 16 v1 tracks: from a low of 0.0% (a bad take where Whisper kept hallucinating "Thank you") to a high of 97.1% (song 3, variant b — near-perfect intelligibility). Seeing that spread at all was the single biggest payoff of adding automated scoring.
After regenerating 7 songs as v2 based on the independent rubric's feedback (one rejection-and-redo), the v1/v2 comparison is where it gets interesting.
- A fix that worked: song 8 went 92.5 → 100.0 (perfect transcription)
- A fix that backfired: song 1 went 75.3 → 53.2. The lyrics got better, but intelligibility dropped. In other words, lyric fixes and seed luck are independent — which is why I switched to "never discard old takes; adopt the best take per song"
- A hands-off ruling: song 3's variant b had a 97.1% match rate (highest of all 16 tracks) and an independent score of 97. Fix suggestions existed, but "risk of touching > benefit," so I declined them. Having the nerve not to touch your highest-scoring take is part of operations too
The final 8 keepers averaged an 83.9% match rate (81.9% at the point of v1 adoption) and 94.1 points on the independent rubric (max 98, min 89 — and the 89-point song had all 4 of its flagged issues fixed).
Being honest about measurement limits
The Whisper match rate is a proxy for pronunciation clarity, not a quality score itself. "Letter singing" (A - B - C) in particular is easy for Whisper to mistranscribe, dragging the rate down (I confirmed mistranscriptions like "EBC" on song 1). Don't cut tracks on the number alone — the final judge is your ears. Bad takes like the 0.0% one do happen, but the sample size is small, so I won't generalize to a "failure rate of X%." The two-variants-per-request design worked as bad-take insurance — this time it absorbed two bad takes at zero extra cost.
7. The recipe (shortest path)
-
Install:
git clone→uv sync→ start the API server (ACESTEP_LM_BACKEND=mlx, the command in §3; run the ~7GB initial download in the background) -
Inspect the endpoint: before generating, verify the endpoint URL and
api_mode(the twoconfig --setlines in §1; don't trust a passing health check) - Generate: caption / structure-tagged lyrics / duration, bpm, key-scale (the command in §4; one request yields two variants)
-
Score: transcribe with mlx-whisper → match rate against the lyrics (the command in §6; pass arm64 Python explicitly via
--python; normalize numbers to English words before comparing) - Final judgment by ear: the match rate is a proxy. Letter-singing dips and the like are Whisper's limits, so decide by listening in the end
8. To my fellow local-AI folks (wrap-up)
Zero cost, zero external calls: 16 vocal tracks plus 14 regenerated ones, all made on this one Mac. Two 100-second songs in about 58 seconds; 8 songs × 2 variants in about 9 minutes (measured on this M1 Max 64GB setup). My answer to "is vocal-song generation on a Mac realistic?" is: on this configuration, generation speed is no longer the problem. What eats your time is lyric design and quality judgment — which is exactly why automating the scoring paid off.
The lessons, in three lines:
-
Don't trust a passing health check. Inspect the endpoint URL and
api_modebefore you say "it runs locally" - Check for arm64 Python first. The x86 landmine comes back even through wrappers like uvx
- Don't grade your own songs. Whisper match rate × independent rubric × ears, in three layers. And lyric fixes and seed luck are independent — never discard old takes
The video installment is here → I Built a 100% Free, Fully Local AI Short-Video Pipeline on an M1 Mac. The series continues, inside this same Mac. Tell me where your local music generation gets stuck sometime, too.
(All numbers and timings are measured on this M1 Max 64GB setup; no speculative numbers, no guarantee wording. License and official-spec citations reflect each official repo/model card as of July 2026.)
Top comments (0)