DEV Community: bigkijimon

graphifyがApple Siliconで全滅した — x86_64 tree-sitterの罠と、コードドキュメントを丸ごと知識グラフ化するまでの実録

bigkijimon — Sun, 12 Jul 2026 02:56:07 +0000

🇯🇵 This post is written in Japanese. Originally published on Zenn（この記事はZennからの転載です）。

Claude Codeが毎回ファイルを読み直す問題

Claude Codeで大きめのプロジェクトを触っていると、毎セッション同じことが起きる。「あのファイルどこだっけ」でgrep、全文Read、また別のファイルをRead……。コードベースはgrepには大きすぎ、コンテキストに丸ごと積むには高すぎる。

これを解くのが graphify（PyPIパッケージ名はgraphifyyで yを2つ重ねる／CLIコマンドはgraphify）。フォルダを丸ごと知識グラフにして、Claudeがファイルを読み直す代わりにグラフを引く。公称「クエリあたり最大71.5xのトークン削減」。

知識グラフとは何かを一言で。コードとドキュメントを解析して「関数・クラス・概念＝ノード」「呼ぶ・依存する・参照する＝有向エッジ」の有向グラフとして抽出したもの。質問すると、全文を読み直す代わりに関連ノードだけを部分グラフで返す。だから軽い。graphifyはコードをtree-sitterでローカルAST解析（LLM不使用）し、ドキュメントだけをモデルで意味抽出する。

M1 Max（Apple Silicon）で、自分のプロジェクト——シェルのオーケストレーション群、Swiftのメニューバーアプリ、設計ドキュメント——を丸ごとグラフ化しようとした。そしていきなり全滅した。この記事はその実録。

トラブルシューティング

全滅：`incompatible architecture`

uv tool install graphifyy で入れて、AST抽出を走らせた瞬間これが出た。

ImportError: dlopen(.../tree_sitter/_binding.cpython-313-darwin.so, 0x0002):
  tried: '.../tree_sitter/_binding.cpython-313-darwin.so'
  (mach-o file, but is an incompatible architecture
   (have 'x86_64', need 'arm64e' or 'arm64'))

tree_sitter のネイティブバインディング（.so）が x86_64 で、arm64のPythonがロードできない。graphifyはコードをtree-sitterでAST解析するので、ここが死ぬとコード抽出が全滅する。

Apple Siliconあるあるだが、厄介なのは「graphify自体が抱えていたx86_64依存」だったこと。自分のコードでもモデルでもなく、ツールのネイティブ依存がx86_64前提でインストールされていた。原因はシンプルで、arm64のPythonがx86_64の.soをdlopenできずに落ちている。uv toolの解決時にx86_64ホイールを掴んでいた。

修正①：tree-sitterをarm64で入れ直す

uv tool環境にはpipが無いので、uv pipでその環境を指定して強制再インストールする。

PY=/Users/you/.local/share/uv/tools/graphifyy/bin/python
uv pip install --python "$PY" --reinstall-package tree-sitter --no-cache 'tree-sitter>=0.23.0'

tree-sitter 0.26.0（arm64）が入り、dlopenの悲鳴は止まった。……が、まだAST結果は0ノードだった。

修正②：言語grammarが「そもそも」入っていなかった

再試行で今度はこう出た。

warning: 32 .swift file(s) contributed nothing to the graph
  because a dependency is missing: tree_sitter_swift not installed.

tree-sitterのコアは直ったが、各言語のgrammar（tree_sitter_swift等）が別パッケージで、未導入だった。ここで一気に入れる。特に tree-sitter-bash を入れると、シェルスクリプトのオーケストレーション群もAST解析できるようになる——これが自分の環境では本命だった。

uv pip install --python "$PY" --no-cache \
  tree-sitter-bash tree-sitter-swift tree-sitter-python \
  tree-sitter-typescript tree-sitter-javascript tree-sitter-json tree-sitter-go

7言語すべてimportOK。このtree-sitter-bash導入が効いて、それまで0ノードだったシェル13本＋Swift32本が解析対象になった。 再々試行：

Swift AST: 323 nodes, 759 edges

動いた。 archを直し、grammarを入れる——この2段で復活する。片方だけだと0ノードで沈黙するので、両方必須。全体の流れはこう。

段階	状態
初期	dlopen `incompatible architecture` で全滅
修正①後（arm64 tree-sitter）	archエラーは消えるが 0ノード
修正②後（言語grammar導入）	332ノード / 653エッジ

コードグラフ構築：bash＋Swiftを1グラフに

コード → 332ノード

まずコード資産をASTで（LLM不使用・完全無料）。シェルのオーケストレーション13本（GPU調停・ディスパッチャ・監視系など）＋Swiftアプリ32本＝45本を1グラフに。

Graph: 332 nodes, 653 edges, 23 communities
God node: 中核ViewModel   ← アプリの中枢を正しく検出

ここでベストプラクティスをひとつ。testは除外、infra/docsは含める。 testはノードを3-4倍に膨らませてクラスタリングを濁す（graphifyの公式ガイドでも指摘される経験則）。逆にインフラ設定やドキュメントは「システムがどう配線されているか（何が何に依存するか）」を教えてくれる。

＋ドキュメント → 629ノード

次に、CLAUDE.mdや設計ドキュメント60本をsemantic抽出（並列サブエージェント）でグラフに足した。結果：

ENRICHED Graph: 629 nodes, 986 edges, 76 communities

コード構造と設計知識が1つのグラフに同居した。構築の流れを図にするとこう。

実際に引いてみる

「GPU調停まわりは何と繋がってる？」を聞くと、BFSでこう返る（識別子は汎用名にしてある。形式は実出力のまま）。

$ graphify query "gpu arbitration freeze thaw"
Traversal: BFS depth=2 | Start: [GpuMonitor] | 13 nodes found

NODE GpuMonitor       [src=app/services/gpu_monitor.swift  L9   community=8]
NODE MemSwitchScript  [src=scripts/mem_switch.sh           L1   community=29]
NODE thaw_ollama()    [src=scripts/mem_switch.sh           L38  community=29]
NODE LocalLLMRuntime  [src=docs/env_snapshot.md            —    community=6]

EDGE GpuMonitor      --shares_data_with--> CoreViewModel   [INFERRED  0.85]
EDGE MemSwitchScript --references-->       LocalLLMRuntime [EXTRACTED 1.0]
EDGE thaw_ollama()   --calls-->            LocalLLMRuntime [EXTRACTED 1.0]

Swiftの型（コード）、シェルのスクリプトと関数（コード）、設計メモの「ローカルLLM基盤」（ドキュメント）が1つの答えに混ざって出てくる。shares_data_with（INFERRED 0.85）やreferences（EXTRACTED 1.0）のように、関係の種類と確信度まで付く。「実装」と「なぜ・どこに繋がるか」を一度に返す——これがコードとドキュメントを同居させた効果だ。graphify explain "<ノード>"ならType / Community / Degreeと接続先一覧を平易に出してくれる。

性能計測：約215xのトークン削減

graphify benchmarkの実測がこれ。

Corpus:          31,450 words → ~41,933 tokens (naive)
Graph:           629 nodes, 986 edges
Avg query cost:  ~195 tokens

表にするとこう。

方式	1回のコスト	倍率
コーパス丸ごと積む（naive）	~41,933 tokens	1x
graphify query	~195 tokens	約215x削減

（コーパス 31,450 words / グラフ 629ノード・986エッジ／graphify benchmark実測）

数字は2つある。公称「最大71.5x」はgraphify公式のベンチ値、この自分のコーパスでの自測値は約215x（graphify benchmarkの出力）。コーパスの内容と質問で変わる——構造・依存を尋ねる質問ほど効き、編集そのものやグラフ外のファイルには効かない。ただ「毎回コードベースを積み直す税」が桁で消えるのは共通だ。

副産物も全部おいしい

graphifyは1回のビルドで大量の成果物を吐く。全部使うと効く。

HTML（graph.html）：ブラウザで開く対話グラフ。サーバ不要。
Obsidian vault：1ノード1mdの双方向リンク。「第二の脳」化。
Wiki：コミュニティ別の記事。エージェントがwiki/index.mdから辿る。
MCPサーバ：python -m graphify.serve graph.json。get_node/get_neighbors/shortest_pathをツール露出。エージェントがグラフをライブ照会できる（要mcpパッケージ）。
常時ON統合：graphify claude installでCLAUDE.mdにルールとPreToolUseフックが入り、以後は明示/graphifyなしでコードベースの質問前に自動でグラフを参照する。

生成されるObsidianノートは1ノード＝1メモで、frontmatterと[[…]]のbacklinkが付く（識別子は汎用名）。まずGpuMonitorのノート：

---
source_file: "app/services/gpu_monitor.swift"
type: "code"
community: "Community 8"
---
# GpuMonitor
## Connections
- [[CoreViewModel]] - `shares_data_with`
- [[MemSwitchScript]] - `conceptually_related_to`

繋がった先のCoreViewModelノートを開くと、逆向きのリンクが自動で張られている（双方向）：

---
source_file: "app/models/core_view_model.swift"
type: "code"
community: "Community 1"
---
# CoreViewModel
## Connections
- [[GpuMonitor]] - `shares_data_with`
- [[MetricsProbe]] - `calls`
- [[SessionProbe]] - `calls`

Obsidianのグラフビューで開けば、この双方向リンクがそのままノード網になり、コードを1行も読まずに「どのモジュールが何と繋がるか」を辿れる。

教訓（Apple Siliconでgraphifyを動かす人へ）

archエラーは2段で直す：tree-sitter本体をarm64で再インストール → 言語grammar（tree-sitter-bash/swift/…）を導入。片方だけだと0ノードで沈黙する。
bash grammarを入れる：シェル中心のオーケストレーションこそグラフ化の価値が高い。
testは切り、infra/docsは入れる：グラフの主役はクラスタの純度と「配線」。
docのsemanticはモデルAPIを使う：コードはローカルAST（無料）、doc/画像はモデル呼び出しでトークンを食う。コスト意識するならスコープを絞る（画像はvisionで特に高い＝依存グラフには不要なので除外が正解）。

archの罠を抜ければ、graphifyはApple Siliconで完全に動く。そして一度グラフにすると、AIが「プロジェクト全体の文脈を持った状態」で動き始める——速度だけの話ではない、というのが一番の収穫だった。

100% Free, Fully Local AI Music Generation on an M1 Mac — ACE-Step 1.5: Two 100-Second Vocal Songs in 58 Seconds (Field Log)

bigkijimon — Sat, 11 Jul 2026 03:35:52 +0000

Zero cost. Zero external calls. On a single M1 Max, two 100-second vocal songs (two seed variants) come out in about 58 seconds (measured on this M1 Max 64GB setup — details below).

The audience I have in mind: Apple Silicon folks running LLMs/image/video models locally, and musicians who want to avoid cloud billing. "Suno is convenient, but I hate the billing and the cloud upload. Does vocal-song generation really work on just a Mac, and how fast is it actually?" — this post answers that with real measurements.

This is the second entry in my "free local AI ___, the honest log" series. Last time it was video → I Built a 100% Free, Fully Local AI Short-Video Pipeline on an M1 Mac. This time it's music. It's the real log of producing eight English learning songs for kids — and I'll start with where I got burned (the failures).

TL;DR

To verify "it's running locally," don't trust a passing health check — inspect the endpoint URL and api_mode. The CLI bundled in the repo points to the cloud (acemusic.ai) by default, and the health check keeps returning OK even while your local server sits there perfectly healthy (the biggest trap; details below).
Measured on an M1 Max 64GB: two variants of a 100-second vocal song = 58.48s (LM 15.70s + DiT 42.78s). Total generation for all 8 songs × 2 variants: about 9 minutes.
Don't judge quality by ear alone: a 3-layer scoring stack — mlx-whisper transcription match rate × an independent rubric × your ears. The final 8 keepers averaged an 83.9% match rate and 94.1 points on the independent rubric.
The environment landmine is the same one as last time (the video post): arm64 Python. This time it was uvx grabbing an anaconda-flavored x86 Python.

1. The incident: "I thought it was local — it was the cloud." Never trust a passing health check

Zero-cost, zero-external-calls AI music — that was the plan. In reality, my generation requests were flying off to the cloud. It's the most valuable failure in this log, so it goes first.

Timeline

Day one. git clone + uv sync, API server up on localhost:8001. Smooth so far.
The repo ships a CLI (acestep.sh) as .claude/skills, and as I started generating with it —
The CLI's default endpoint was the cloud API (acemusic.ai). Even with a perfectly healthy API server running on my machine, requests were configured to go external.

Why you won't notice: the cloud health check returns OK

Here's the nasty part. The CLI's health check returns "OK" — as a health check of the cloud side. Your local server also started fine. So "server up OK, health check OK, generation works" — from every angle it looks like "it works," while the reality is external transmission. The more strictly you require fully-local (zero external calls), the more this cosmetic OK will fool you.

The fix is two lines

Point the endpoint and the API mode explicitly at local:

bash scripts/acestep.sh config --set api_url "http://127.0.0.1:8001"
bash scripts/acestep.sh config --set api_mode native

Lesson: If your requirement is fully local (zero external calls, commercial-safe), don't accept "health check OK" — inspect the endpoint URL and api_mode yourself. A passing health check and requests staying local are two different questions.

Every measurement below was taken after this fix (api_mode native, targeting 127.0.0.1).

2. Why ACE-Step 1.5 — model selection from a Mac perspective

I compared the usual "local AI music on a Mac" candidates along three axes: vocals (songs with singing), runs on a Mac, and license. Every license claim below is primary-sourced from the official repo/model card (as of July 2026).

Model	Why I passed	License (primary-sourced)
MusicGen / AudioCraft	Weights are CC-BY-NC 4.0 = no commercial use (code is MIT). Also, vocals were removed from the training data via source separation, and the official card states "The model is not able to generate realistic vocals." = can't make songs	Code=MIT / weights=CC-BY-NC 4.0, model card
YuE	No Mac path. The official repo states "FlashAttention 2 is mandatory" = CUDA-first, with no macOS/Apple Silicon support mentioned. (License itself is Apache 2.0, which is fine)	Apache 2.0
Stable Audio Open	The official model card states "The model is not able to generate realistic vocals." = no vocal songs	Stability AI Community License
DiffRhythm	Code and DiT weights are Apache 2.0, but the VAE alone is under the Stability AI Community License (officially: "DiffRhythm-VAE is subject to the Stability AI Community License Agreement") — licensing splits per component. Messy to manage	Code/DiT=Apache 2.0, VAE=Stability AI Community License

By the way, MusicGen — which I used for BGM in the previous post — is, per the table, "non-commercial + can't sing," so it stopped at BGM. Needing a different model for actual songs is where this project started.

What sold me on ACE-Step 1.5 — three things:

MIT license (no strings for commercial use; compared with the table above, that simplicity is precious)
Official macOS/Apple Silicon support. A start_api_server_macos.sh ships in the repo; the LM side runs on MLX and the DiT side on PyTorch MPS
A bundled REST API server, easy to wire into batch production and automation

The LM comes in 0.6B/1.7B/4B. I started with 0.6B. There's an external report of 1.7B peaking at 42GB (not verified by me), so on a machine that also keeps other local LLMs resident, skipping it felt like the safe call.

3. Install and launch (M1 Max 64GB, ~7GB initial download)

The install itself is straightforward, so I'll keep this short.

Environment: MacBook Pro M1 Max 64GB (unified memory). On startup the server detected "51.8GB unified memory, tier=unlimited". Models: LM = acestep-5Hz-lm-0.6B + DiT = acestep-v15-turbo.

# Start the API server (port 8001, LM on the MLX backend)
ACESTEP_LM_BACKEND=mlx TOKENIZERS_PARALLELISM=false \
  uv run acestep-api --host 127.0.0.1 --port 8001 --lm-model-path acestep-5Hz-lm-0.6B

# Confirm it's listening
lsof -nP -iTCP:8001 -sTCP:LISTEN

The first run downloads the models. Measured: about 7GB total under checkpoints/ (about 4 minutes on a 47MB/s connection).

Three small traps:

start_api_server_macos.sh can pop an interactive prompt during its git update check. Calling uv run acestep-api directly is the reliable path.
The bundled CLI scripts/acestep.sh doesn't have its executable bit set. Call it via bash.
The initial download and long generations will time out if you run them synchronously. Run them in the background.

4. Measured: two 100-second vocal songs in 58.48s (LM 15.70s + DiT 42.78s)

The main event. Generation is just handing the CLI a caption (style tags), lyrics with structure tags, and parameters.

bash scripts/acestep.sh generate -c "<caption>" -l "<full lyrics with structure tags>" \
  --duration 100 --bpm 100 --key-scale "C major" --language en

Here's the result metadata for song 1 (a 100-second ABC song). One request auto-generates two variants with different seeds, and the recorded time is the total for both.

"generation_info": "**🎵 Total generation time (2 songs): 58.48s**
  - 29.24s per song
  - LM phase (2 songs): 15.70s
  - DiT phase (2 songs): 42.78s",
"seed_value": "1014051477,117013642"

Two finished 100-second songs in 58.48s. Generation runs faster than the song's actual length — faster than real time. The breakdown: LM (lyrics → semantic tokens) 15.70s, DiT (audio synthesis) 42.78s. Having this breakdown tells you which side to lighten if you want more speed, so I recommend keeping every meta.json.

Measurements across all 8 songs

That pace wasn't first-song luck; it held across all eight.

Song	Length	Generation time (2 variants)
01	100s	58.48s
02	110s	66.32s
03	—	62.99s
04	—	63.97s
05	—	68.92s
06	—	65.55s
07	120s	74.91s
08	115s	74.90s

Total generation time for all 8 songs × 2 variants (16 tracks): about 9 minutes. The outputs are 100–125s mp3s at 48kHz/128kbps, roughly 1.6MB (100s) to 2.0MB (125s) each. The lyrics were synthesized word-for-word as written (verified by inspecting metas.lyrics in meta.json).

All numbers are measured on this M1 Max 64GB configuration. I haven't measured M2/M3/M4, so I'm not extrapolating to other chips. This is not a promise of "58 seconds for everyone."

5. Lyric design: the LMNOP problem is universal

Fast generation is useless if the lyric design is off. From here on, this is operational field experience you won't find in the usual roundups.

The "elemenopee" problem

In the traditional ABC-song melody, L-M-N-O-P gets crammed together and smears into "elemenopee" — a well-known issue. I avoided it by regrouping the letters into evenly spaced chunks: ABCD / EFG / HIJK / LMN / OPQ / RST / UVW / XYZ.

This isn't just about the alphabet — it's a principle for any enumeration lyric (numbers, days of the week): place each item at even intervals and don't insert "and" into the enumeration ("twenty-nine and thirty" was actually flagged in scoring).

General techniques for clearer AI singing

Lyric-side techniques that solidified in practice, and should apply beyond ACE-Step:

Write numbers as English words ("30" → "thirty"; the transcription scoring needed number normalization too)
Have letters sung one at a time with separators: A - B - C
4–8 words and 6–10 syllables per line; keep parallel lines within ±1–2 syllables
Repeat choruses with identical spelling, word for word. Use parentheses for backing vocals (happy!) and structure tags like [call and response]
Keep the caption to 4–7 consistent tags, and never let caption and lyric moods contradict. Don't put BPM/key in the caption — pass them as dedicated parameters
Three prohibitions that came out of independent scoring: 1) no "and" inside enumerations; 2) don't contort English for the sake of rhyme ("but fun, all right!" was a real example); 3) at most one alteration per chorus, and keep the same enumeration consistent within a song (no mixing fall vs autumn)

The real thing: song 1's full lyrics and parameters

Parameters: duration 100s / BPM 100 / C major / vocal en
caption: children's song, cheerful, playful, simple melody, ukulele, glockenspiel, acoustic guitar, clear female vocals, warm, sing-along

[Intro]

[Verse]
A - B - C - D
E - F - G
H - I - J - K
L - M - N

[Verse]
O - P - Q
R - S - T
U - V - W
X - Y - Z

[Chorus]
Now I know my A - B - C (A - B - C!)
Sing the alphabet with me (sing with me!)
Happy letters, you and me
Come and sing the A - B - C

[Verse] (repeat)
[Chorus] (repeat)

[Outro]
A - B - C, you and me!

These lyrics, as-is, became the two variants in the 58.48s above (measured 48kHz, 100.03s each).

6. "Don't grade your own songs" — the 3-layer Whisper scoring stack

Comparing 16 tracks entirely by ear to rank them is, honestly, brutal. So I inserted objective scoring via transcription — and stepped on another landmine setting it up, so let's start there.

Failure #2: uvx grabs an x86 Python

I tried to install mlx-whisper (fully local transcription) for scoring via uvx, and dependency resolution failed. The cause: this machine's default python3 is an anaconda-provided x86_64 (Rosetta) build, and uvx grabbed it. mlx is arm64-only, so it won't install.

It's the sequel to the "make it grab arm64 Python" landmine from the previous post. You can avoid it when you create venvs yourself — but it comes back through convenience wrappers like uvx. Passing an arm64 CPython explicitly via --python gets you through:

uvx --python cpython-3.12-macos-aarch64-none --from mlx-whisper \
  mlx_whisper <mp3> --model mlx-community/whisper-large-v3-turbo

When an MLX tool fails mysteriously on Apple Silicon, first check your arch with python3 -c "import platform; print(platform.machine())" — especially on machines that also have anaconda.

The three layers

Objective layer: transcribe each track with mlx-whisper (large-v3-turbo, Apache 2.0, fully local) and compute the word-sequence match rate against the lyrics actually used for synthesis (verified from meta.json). Normalize numbers to English words before comparing (to correct false-low scores from "1,2,3"-style transcriptions)
Independent layer: have an agent in a separate context score against a rubric (English correctness 25 / design-principle adherence 25 / educational value 25 / singability 25 = 100 points). If the author (the same-context session) grades it, the grades go soft — so separate them
Final layer: human ears. A person covers Whisper's measurement limits (below)

What the data actually said

Match rates spread far more than I expected. Across the 16 v1 tracks: from a low of 0.0% (a bad take where Whisper kept hallucinating "Thank you") to a high of 97.1% (song 3, variant b — near-perfect intelligibility). Seeing that spread at all was the single biggest payoff of adding automated scoring.

After regenerating 7 songs as v2 based on the independent rubric's feedback (one rejection-and-redo), the v1/v2 comparison is where it gets interesting.

A fix that worked: song 8 went 92.5 → 100.0 (perfect transcription)
A fix that backfired: song 1 went 75.3 → 53.2. The lyrics got better, but intelligibility dropped. In other words, lyric fixes and seed luck are independent — which is why I switched to "never discard old takes; adopt the best take per song"
A hands-off ruling: song 3's variant b had a 97.1% match rate (highest of all 16 tracks) and an independent score of 97. Fix suggestions existed, but "risk of touching > benefit," so I declined them. Having the nerve not to touch your highest-scoring take is part of operations too

The final 8 keepers averaged an 83.9% match rate (81.9% at the point of v1 adoption) and 94.1 points on the independent rubric (max 98, min 89 — and the 89-point song had all 4 of its flagged issues fixed).

Being honest about measurement limits

The Whisper match rate is a proxy for pronunciation clarity, not a quality score itself. "Letter singing" (A - B - C) in particular is easy for Whisper to mistranscribe, dragging the rate down (I confirmed mistranscriptions like "EBC" on song 1). Don't cut tracks on the number alone — the final judge is your ears. Bad takes like the 0.0% one do happen, but the sample size is small, so I won't generalize to a "failure rate of X%." The two-variants-per-request design worked as bad-take insurance — this time it absorbed two bad takes at zero extra cost.

7. The recipe (shortest path)

Install: git clone → uv sync → start the API server (ACESTEP_LM_BACKEND=mlx, the command in §3; run the ~7GB initial download in the background)
Inspect the endpoint: before generating, verify the endpoint URL and api_mode (the two config --set lines in §1; don't trust a passing health check)
Generate: caption / structure-tagged lyrics / duration, bpm, key-scale (the command in §4; one request yields two variants)
Score: transcribe with mlx-whisper → match rate against the lyrics (the command in §6; pass arm64 Python explicitly via --python; normalize numbers to English words before comparing)
Final judgment by ear: the match rate is a proxy. Letter-singing dips and the like are Whisper's limits, so decide by listening in the end

8. To my fellow local-AI folks (wrap-up)

Zero cost, zero external calls: 16 vocal tracks plus 14 regenerated ones, all made on this one Mac. Two 100-second songs in about 58 seconds; 8 songs × 2 variants in about 9 minutes (measured on this M1 Max 64GB setup). My answer to "is vocal-song generation on a Mac realistic?" is: on this configuration, generation speed is no longer the problem. What eats your time is lyric design and quality judgment — which is exactly why automating the scoring paid off.

The lessons, in three lines:

Don't trust a passing health check. Inspect the endpoint URL and api_mode before you say "it runs locally"
Check for arm64 Python first. The x86 landmine comes back even through wrappers like uvx
Don't grade your own songs. Whisper match rate × independent rubric × ears, in three layers. And lyric fixes and seed luck are independent — never discard old takes

The video installment is here → I Built a 100% Free, Fully Local AI Short-Video Pipeline on an M1 Mac. The series continues, inside this same Mac. Tell me where your local music generation gets stuck sometime, too.

(All numbers and timings are measured on this M1 Max 64GB setup; no speculative numbers, no guarantee wording. License and official-spec citations reflect each official repo/model card as of July 2026.)

I Built a 100% Free, Fully Local AI Short-Video Pipeline on an M1 Mac — For Fellow Local-Claude Folks

bigkijimon — Fri, 10 Jul 2026 06:35:51 +0000

Zero cost. Zero external calls. Draw keyframes with SDXL, bring them to life with LTX-2.3 (MLX) image-to-video, narrate with Kokoro TTS, score it with MusicGen, and stitch it all together with ffmpeg — entirely inside one M1 Max (64GB). This is the real log, plus a recipe you can copy.

Audience: Claude Code users and developers running Claude/LLMs locally on Apple Silicon. I wrote down every place I got stuck, because the failures are the most useful part.

TL;DR

Panning/zooming a still image (Ken Burns) is not a video. It's a slideshow. Real motion only comes from I2V (image-to-video). Not grasping this first was my biggest mistake.
The whole stack is free and local: SDXL (ComfyUI) keyframes → LTX-2.3 (MLX) I2V → Kokoro TTS narration → MusicGen BGM → ffmpeg assembly.
The nastiest gotcha is the Metal GPU watchdog crash (MLX issue #3267). The old package died instantly while the display was on; the newer dgrauet/ltx-2-mlx (q8) runs to completion with the screen on — that reversal was the biggest lesson.
Environment setup is where 99% of people get stuck: arm64 Python and pinning transformers.

1. How it started: "This is just a slideshow"

My first delivery was rejected in one sentence.

I generated ten nice stills with SDXL, faked motion with ffmpeg's zoompan (the Ken Burns push/pull), laid narration and subtitles on top, and called it "a video." The response:

"This isn't what I wanted. It's just a slideshow. I asked for a video, not a slideshow."

It stung. Here's what I burned into my bones:

A "video" (movie) means the characters, the water, the light actually move. A zoom/pan over a still is not a video. Never conflate the two.
Pin down the definition of the deliverable first. Don't sacrifice what the client actually wants (real motion) for "cheap and fast." Cheap and fast, but wrong, is worth zero — it's a full redo.
Real motion requires I2V. A stills-only pipeline can only get you to a "storyboard / preview."
But don't throw Ken Burns away — it's great for matching narration length in storyboard checks, and for closing shots that don't need to move. Just never call that "finished."

That single rejection was the origin of everything below.

Side-by-side, from the actual project. Left = the rejected slideshow (zoom on a still). Right = real I2V motion from the same keyframe.

2. The goal: run real motion (I2V) locally, for free

The idea is simple. Make a still image (a keyframe) for each shot, then have an I2V model bring that keyframe to life. Keyframes come from SDXL; the motion comes from the MLX port of LTX-2.3. All of it runs inside this Mac.

Fig 1: The whole pipeline — I2V (LTX-2.3) is the body of the video

Stage	Step	Tool	Role
A	Storyboard + narration script	Local LLM (gemma-3-12b-it via mlx_lm)	Design
B	Keyframe stills	ComfyUI / SDXL (realcartoon-xl)	Starting image per shot
C	Narration audio + real duration	Kokoro TTS (or macOS `say`) + `ffprobe`	Free; locks timing
D	I2V — "move" the keyframe	LTX-2.3 MLX	This is the body of the "video." Non-skippable
E	Assembly (concat + narration + subs + BGM)	ffmpeg	Finishing
(preview only)	Ken Burns	ffmpeg zoompan	For review only. Not the finished product

3. The stack (all free, all local)

Keyframes: ComfyUI + realcartoon-xl-v4 (SDXL family). Batch-generate by hitting the REST API (:8000).
I2V (the video body): MLX port of LTX-2.3, in two flavors by purpose (below).
- Draft = notapalindrome/ltx23-mlx-av-q4 (20GB, distilled q4, fast)
- Production = dgrauet/ltx-2.3-mlx-q8 (21GB, --two-stages-hq + STG, high fidelity)
Text encoder / script LLM: mlx-community/gemma-3-12b-it-4bit (~7GB).
Narration: Kokoro TTS (mlx-audio). macOS's built-in say works too.
BGM: MusicGen (transformers' built-in MusicgenForConditionalGeneration).
Assembly: ffmpeg (Homebrew build — with a caveat below).

Hardware is M1 Max / 64GB. That "64GB reality" constrains model choice.

4. Environment setup is where you'll get stuck (two real landmines)

Honestly, setup is harder than the generation itself. 99% of the pain is here. Two landmines:

1) Use arm64 Python (Anaconda will kill you)

Anaconda-style Python often runs as x86_64 (Rosetta), and there mlx simply won't install (no matching distribution). Always create your venv with native arm64 Python.

/Library/Frameworks/Python.framework/Versions/3.12/bin/python3 -m venv ~/ltx-mlx
~/ltx-mlx/bin/pip install -U mlx-video-with-audio

The newer package (dgrauet/ltx-2-mlx) uses uv, but uv may also grab an x86_64 anaconda python at first and fail to install mlx, so pass the arm64 python explicitly:

git clone --depth 1 https://github.com/dgrauet/ltx-2-mlx.git ~/ltx-2-mlx
cd ~/ltx-2-mlx
uv sync --all-extras --python /Library/Frameworks/Python.framework/Versions/3.11/bin/python3

2) Pin transformers to `>=4.50,<5`

pip install mlx-video-with-audio drags in transformers 5.13, which crashes mlx_lm's tokenizer registration (AttributeError: 'str' object has no attribute '__module__'). Pin it down and it works (4.57.6 runs fine; pip warns about deps but it works):

~/ltx-mlx/bin/pip install "transformers>=4.50,<5"   # ← the required pin

Miss these two and you'll lose hours to "mlx won't install for some reason" and "a mysterious AttributeError on startup." Kill them first.

5. Model choice and the "64GB reality"

The full-precision LTX-2.3 model is 53.5GB as a single file, which OOMs / thrashes on 64GB at load. Don't use it. Run quantized versions.

Fig 2: Which quantized build to use, inside the 64GB reality

Purpose	Model / settings	Measured speed
Speed / draft	distilled q4 (`ltx23-mlx-av-q4` / `--distilled`)	~4 min/clip, peak ~14.7GB
Quality / production	q8 `--two-stages-hq` + STG	~30 min/clip, peak ~21GB
Fights / heavy motion	q8 HQ + `--stg-scale 1.0` + motion LoRA	30 min+

For HQ at 1024×576, 97 frames, it's ~30 min/clip (1774s) — of which Stage 1 is 24 min (15 steps × ~95s). That's ~7.5× slower than distilled q4 (~4 min), but bricks, water, and edges come out clearly sharper. Iterate drafts on q4, then bake only the final on q8 — that was the practical answer.

6. The recipe (the actual commands I ran)

1) Script (local LLM, fully autonomous)

mlx_lm + gemma-3-12b-it-4bit generates the shot list, per-shot visual prompts, and narration together.

"$PY/mlx_lm.generate" --model mlx-community/gemma-3-12b-it-4bit \
  --max-tokens 1100 --temp 0.75 --prompt "$PROMPT" \
  | sed '/^==========/d' > script.md

The prompt enforces "a strong hook in the first 3 seconds," "visual prompts in English describing people/place/light/motion," and "no speculative numbers, no exaggeration, no guarantee wording."

2) Keyframes (ComfyUI / SDXL)

POST a graph to ComfyUI's REST (:8000) to get stills. Sampler: dpmpp_2m / karras / steps 30 / cfg 6.5.

"3": {"class_type": "KSampler", "inputs": {
    "seed": seed, "steps": 30, "cfg": 6.5,
    "sampler_name": "dpmpp_2m", "scheduler": "karras", "denoise": 1.0,
    ...}}

Character consistency is the lifeline, so reuse a fixed design description of the main characters across every prompt. SDXL loves to add extra people, so put three people, group in the negative prompt to hold it back.

A real keyframe generated with ComfyUI/SDXL (the starting image of cut01)

Character consistency held via a fixed design description (cut07, third retake)

3) I2V (the video body — LTX-2.3)

Draft = q4 (~4 min/shot). Run it alone, with caffeinate to block sleep.

caffeinate -i -s ~/ltx-mlx/bin/python -m mlx_video.generate_av \
  --model-repo notapalindrome/ltx23-mlx-av-q4 \
  --text-encoder-repo mlx-community/gemma-3-12b-it-4bit \
  --image kf.png --prompt "your motion, cinematic film footage, natural gentle motion, warm golden light" \
  --negative-prompt "static, morphing, warping, extra fingers, flicker, jitter, low quality" \
  -W 1024 -H 576 --num-frames 65 --fps 24 --no-audio \
  --output-path cut.mp4

Production = q8 HQ (~30 min/shot, high fidelity).

/Users/yuma/ltx-2-mlx/.venv/bin/ltx-2-mlx generate \
  --model dgrauet/ltx-2.3-mlx-q8 --two-stages-hq \
  --image keyframe.png 0 1.0 \
  --prompt "your motion, cinematic film, natural motion, warm golden light" \
  --frames 97 --frame-rate 24 -H 576 -W 1024 \
  --stg-scale 1.0 --cfg-scale 4.0 --seed 42 \
  --output cut.mp4

A real generated cut from this pipeline (GIF-compressed for the article; the original is 1024×576, 24fps, with audio)

Key notes:

Resolution must be a multiple of 64 (1024×576 recommended = true 16:9). For vertical Shorts, -H 1024 -W 576.
num-frames is 8n+1 (max 97 on the q4 family). Distilled models ignore --steps (fixed at 8+3).
--no-audio drops the model's audio so you can add your own narration afterward (the right call for educational content).
--image PATH FRAME STRENGTH lets you anchor both end frames, interpolating longer continuous motion.

4) Narration (Kokoro TTS)

~/ltx-mlx/bin/python -m mlx_audio.tts.generate \
  --model mlx-community/Kokoro-82M-bf16 \
  --text "English narration" --voice bm_george --lang_code b \
  --output_path seg_01.wav
# Get real duration: ffprobe -v error -show_entries format=duration -of csv=p=0 seg_01.wav

You'll need pip install "misaki[en]". If a particular sentence throws a shape error, tweak the wording and retry. Kokoro is light and doesn't trip the GPU watchdog, so it's fine with the screen on. macOS's say -v Daniel -r 165 works in a pinch too (free, reliable).

5) BGM (MusicGen)

audiocraft chokes on the av / pkg-config build, so skip it — the built-in MusicGen in transformers is the answer. facebook/musicgen-small on CPU (~4.6 min for 20s).

from transformers import AutoProcessor, MusicgenForConditionalGeneration
model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")
prompt = "warm gentle emotional cinematic film score, soft solo piano with tender strings, slow, heartfelt, in a major key"
audio = model.generate(**inputs, do_sample=True, guidance_scale=3.0, max_new_tokens=1024)  # ~20s

Loop the 20s with acrossfade to match length, and mix it in gently at volume=0.30.

6) Assembly (ffmpeg)

Upscale each shot to 1080p (lanczos + a light grade), burn the subtitles, add narration, concat all shots, and amix the BGM at the end. Normalize loudness with loudnorm=I=-15. Burn subtitles per shot so they never drift from the narration.

7. The full gotcha list (the main event for developers)

This is what I most want to share. I stepped on all of these so you don't have to.

★ Metal GPU watchdog crash (MLX issue #3267)

Fig 3: The watchdog mechanism, and the reversal with the new package

The symptom:

[METAL] Command buffer execution failed: Impacting Interactivity
(kIOGPUCommandBufferCallbackErrorImpactingInteractivity)

The eerie behavior: only the first LTX run after a reboot succeeds; the second onward dies instantly.

The root cause is confirmed in MLX issue #3267: while the display is on, WindowServer composites the screen on the GPU; when MLX's heavy command buffer blocks that compositing, macOS kills the process to "preserve interactivity." Display on = 100% reproducible; off = 100% avoided.

What did NOT work:

caffeinate -i -s — prevents system sleep, but doesn't turn off the display, so it's useless here.
pmset displaysleepnow — if an external monitor stays alive, compositing continues; no good.
Stopping ComfyUI / /free — unrelated.

For the old package (mlx-video-with-audio distilled q4), the reliable fix was: 1) reboot the Mac to clear the post-hang driver leak, 2) power off the external monitor and close the lid so nothing is displayed, 3) run the LTX batch in that state. That got all 10 shots through with zero crashes.

But the newer package overturned this. dgrauet/ltx-2-mlx (q8 HQ) generated a shot with the screen left on → 30.8 min, zero crashes, no new crash report. So you can run HQ batches in the background during the day while a human uses the screen. No more being chained to unattended overnight jobs. It was a nice moment to walk back the earlier "screen must be off" conclusion with real measurements (do keep ventilation, though — it runs hot).

★ Homebrew ffmpeg has no libass → subtitles won't burn

The Homebrew ffmpeg is built without libass/freetype, so neither the subtitles nor the drawtext filter exists (if ffmpeg -filters | grep subtitles is empty, that's you). You can make an SRT, but you can't burn it.

The workaround: draw subtitles onto a transparent PNG with Pillow, then burn them with ffmpeg's overlay. Font: /System/Library/Fonts/Supplemental/Arial Bold.ttf. Use a thick, dark stroke and lay a semi-transparent rounded plate behind the text for readability.

★ Boomerang (reverse-loop) artifacts

An LTX clip is ~2.7–4s (97 frames). If you pad a long-narration shot with "forward + reverse (boomerang)," you get reverse artifacts — things move and then un-move (e.g. someone "un-sips" a drink). That's the single biggest deduction on motion quality.

The fix: don't reverse. Hold the last frame as a clone and add a gentle drift:

tpad=stop_mode=clone:stop_duration=X   # clone-hold the final frame out to the narration length
zoompan=z='min(1.0+0.0006*on,1.11)':d=1:fps=24   # imperceptible drift

If you need genuinely longer continuous motion, chain LTX clips (last frame of a clip → next keyframe → generate → concat).

★ 4x-UltraSharp (ESRGAN) breaks on Apple MPS

Using ESRGAN-family upscalers (4x-UltraSharp) produces scanline-style breakup on Apple's MPS. Don't. If you need 4K, a clean ffmpeg upscale (scale=3840:2160:flags=lanczos + a light unsharp) looks perfectly fine.

★ The scratchpad (/tmp) is volatile

Put working scripts in /tmp and they vanish on reboot/cleanup. Keep important scripts in your deliverables folder. A humble lesson that pays off.

★ LoRA turned out to be possible (retraction)

I originally concluded "LoRA is Wan-only, LTX doesn't support it, it's a dead end." That was wrong. I confirmed support in dgrauet/ltx-2-mlx generate --help:

--lora PATH STRENGTH — a normal LoRA (motion boost, speed LoRA, etc.)
--distilled-lora / --distilled-lora-strength — LoRA for the distilled (fast) path
--stg-scale composes with --lora (per-block application)

So the "LTX + multiple LoRAs" route is real. You don't have to touch ComfyUI-PyTorch (high risk of breaking your torch install) — you can attach LoRAs straight from the MLX CLI. For fight-level heavy motion, the main line is "q8 --two-stages-hq + --stg-scale 1.0 + a motion LoRA."

8. Quality-score progression (the honest log)

Fig 4: The honest climb — still short of the 95-point gate

Internally, videos go through independent scoring (the author doesn't grade their own; a separate session does) with a 95-point gate. Here's how the actual score moved:

v1 slideshow → rejected
 → Wan (I2V) 70
 → LTX character-consistent 76 → 77
 → all shots real LTX 78
 → all boomerangs removed 81
 → + Kokoro / + MusicGen / subtitle plate / -15 LUFS ≈ 85 (estimated)

From slideshow → real video → ~85, still free. Not at 95 yet (motion length and timing are the ceiling). I'll be honest about that. The next moves are LTX chaining and "HQ + motion LoRA" to push toward 90. The accurate framing isn't "free and local is perfect" — it's "free and local got us this far."

9. The recipe (shortest path)

venv with arm64 python (don't grab Anaconda's x86_64) → install mlx-video-with-audio, pin transformers>=4.50,<5.
Script — mlx_lm.generate + gemma-3-12b-it-4bit for shot list + visual prompts + narration.
Keyframes — ComfyUI (:8000) + realcartoon-xl-v4. Reuse a fixed character design across shots; suppress extra people via the negative prompt.
I2V — draft on q4 (generate_av, ~4 min), production on q8 --two-stages-hq + STG (~30 min). Resolution a multiple of 64, frames 8n+1, --no-audio.
Narration — Kokoro TTS (mlx_audio / misaki[en]). Get real duration with ffprobe.
BGM — transformers' built-in MusicGen (facebook/musicgen-small, CPU). Mix gently at volume=0.30.
Assembly — ffmpeg to 1080p + grade, subtitles via Pillow transparent PNG → overlay (libass workaround), no reverse — clone-hold + drift, then amix the BGM + loudnorm=I=-15.
Time-shift heavy GPU work (colliding with other work on the same Mac slows or crashes both). For long jobs, caffeinate -i + good airflow.

10. To everyone running Claude locally

A message and some tips for developers running Claude/LLMs locally, like me.

You don't have to pay — one Apple Silicon machine gets you this far. Cloud I2V (Kling/Runway/Veo…) is gorgeous and fast, but it means external calls and billing. Keeping everything on your own Mac is, beyond cost, mostly about the calm of holding all the pieces yourself. When you get stuck, the cause is entirely inside your own environment.

Tips to dodge the pitfalls:

Don't melt hours on setup. 80% of "mlx won't install" and "mysterious AttributeError on startup" is either "you grabbed an x86_64 python" or "transformers is 5.x." Kill those two first and the rest is the fun part.
Keep asking "is it actually moving?" Don't settle for a zoom over a pretty still. It's not a video until it goes through I2V. Cheap and fast but wrong is worth zero.
Turn failure logs into assets. Save every winning prompt + seed, every model's runtime, every landmine into knowledge/. Reproducibility is the asset. Our score climbed 77→85 precisely because we turned each failure into one line of notes.
Time-shift heavy GPU tasks. Run them alongside other heavy work or your own session on the same Mac and both slow down or crash. There's one GPU. Batch before bed or while away.
Be proud of "this far, for free." It doesn't have to be perfect. Slideshow → real motion, one step at a time. Getting to share that log with each other is the best part of being local-Claude folks.

I'll make another one, inside this same Mac. Tell me where your pipeline gets stuck sometime, too. 🌿

(All figures and timings are measured on this M1 Max 64GB setup. No speculative numbers, no guarantee wording.)

DEV Community: bigkijimon

graphifyがApple Siliconで全滅した — x86_64 tree-sitterの罠と、コード ドキュメントを丸ごと知識グラフ化するまでの実録

Claude Codeが毎回ファイルを読み直す問題

トラブルシューティング

全滅：incompatible architecture

修正①：tree-sitterをarm64で入れ直す

修正②：言語grammarが「そもそも」入っていなかった

コードグラフ構築：bash＋Swiftを1グラフに

コード → 332ノード

＋ドキュメント → 629ノード

実際に引いてみる

性能計測：約215xのトークン削減

副産物も全部おいしい

教訓（Apple Siliconでgraphifyを動かす人へ）

100% Free, Fully Local AI Music Generation on an M1 Mac — ACE-Step 1.5: Two 100-Second Vocal Songs in 58 Seconds (Field Log)

TL;DR

1. The incident: "I thought it was local — it was the cloud." Never trust a passing health check

Timeline

Why you won't notice: the cloud health check returns OK

The fix is two lines

2. Why ACE-Step 1.5 — model selection from a Mac perspective

3. Install and launch (M1 Max 64GB, ~7GB initial download)

4. Measured: two 100-second vocal songs in 58.48s (LM 15.70s + DiT 42.78s)

Measurements across all 8 songs

5. Lyric design: the LMNOP problem is universal

The "elemenopee" problem

General techniques for clearer AI singing

The real thing: song 1's full lyrics and parameters

6. "Don't grade your own songs" — the 3-layer Whisper scoring stack

Failure #2: uvx grabs an x86 Python

The three layers

What the data actually said

Being honest about measurement limits

7. The recipe (shortest path)

8. To my fellow local-AI folks (wrap-up)

I Built a 100% Free, Fully Local AI Short-Video Pipeline on an M1 Mac — For Fellow Local-Claude Folks

TL;DR

1. How it started: "This is just a slideshow"

2. The goal: run real motion (I2V) locally, for free

3. The stack (all free, all local)

4. Environment setup is where you'll get stuck (two real landmines)

1) Use arm64 Python (Anaconda will kill you)

2) Pin transformers to >=4.50,<5

5. Model choice and the "64GB reality"

6. The recipe (the actual commands I ran)

1) Script (local LLM, fully autonomous)

2) Keyframes (ComfyUI / SDXL)

3) I2V (the video body — LTX-2.3)

4) Narration (Kokoro TTS)

5) BGM (MusicGen)

6) Assembly (ffmpeg)

7. The full gotcha list (the main event for developers)

★ Metal GPU watchdog crash (MLX issue #3267)

★ Homebrew ffmpeg has no libass → subtitles won't burn

★ Boomerang (reverse-loop) artifacts

★ 4x-UltraSharp (ESRGAN) breaks on Apple MPS

★ The scratchpad (/tmp) is volatile

★ LoRA turned out to be possible (retraction)

8. Quality-score progression (the honest log)

9. The recipe (shortest path)

10. To everyone running Claude locally

graphifyがApple Siliconで全滅した — x86_64 tree-sitterの罠と、コードドキュメントを丸ごと知識グラフ化するまでの実録

全滅：`incompatible architecture`

2) Pin transformers to `>=4.50,<5`