DEV Community: GeneLab_999

I didn't fix the bug: contributing to a 20k-star ML repo by measuring it

GeneLab_999 — Thu, 24 Sep 2026 04:41:45 +0000

Last week I opened an issue on laya, an open-source decision model with a little over 20k stars. The issue reported that one of its checkpoints almost never picks the first-listed option on ordinal questions. About 36 hours later, three things had happened. The maintainer had documented the limitation in a release. He had asked me to run the before/after on the retrained checkpoint. And he had merged a regression check I wrote. That was four days after my first measurement.

Nothing is fixed. The fix is a retrain, and the retrain hasn't happened yet. What I contributed is narrower. I think it is also the kind of thing an outsider is best placed to contribute: a measurement nobody could wave away, and a tool that says whether the fix worked.

The fastest way I've found to contribute to an ML repo you don't maintain: measure it, make the measurement impossible to explain away, and then hand the maintainer a tool.

The timeline, in UTC:

Sep 20 — first Japanese baseline run
Sep 22, 05:56 — issue #131 opened
Sep 22, 06:40 — condition F posted (position vs. label, explained below)
Sep 22, 07:15 — a third party, AlKor13, traces it to the weights
Sep 22, 10:56 — my labelled A/B test of his hypothesis
Sep 22, 20:40 — the maintainer's decision
Sep 23, 07:25 — v0.3.7 ships with the limitation in the README
Sep 23, 14:42 — PR #259 opened
Sep 23, 17:48 — PR #259 merged

What laya is, and why I was measuring it

laya is a non-autoregressive "System 1" decision model. You give it some text and a set of typed questions: choice, score for ordinal levels, and noul for yes/no. It answers all of them in one forward pass, with probabilities and no text generation. It ships several checkpoints, including an English laya and a laya-multilingual.

I'm building a Japanese model in the same space (sokudan). Before writing any training code, I wanted a baseline: how well does the existing multilingual checkpoint handle Japanese?

I built 300 label-conditioned Japanese business emails. The labels were fixed first. Then a local LLM wrote an email to match them, and any email containing the label words themselves was thrown out and regenerated. Each email got three questions: department (4-way choice), urgency (3-level score) and whether the customer is hinting at cancelling (bool).

The results split by question type. choice worked: 0.747 accuracy against a 0.380 majority-class baseline. score lost to the majority baseline (RPS 0.232 vs 0.197, lower is better). bool also lost to the majority baseline (0.543 vs 0.703), with an AUROC of 0.523. That is barely better than ranking at random.

One comparison I did not run is TypeSafe's Jev, the hosted API laya benchmarks itself against. Its customer agreement prohibits using the service or its outputs to develop similar products, and my project is a similar product.

The number that didn't fit

"Worse than majority" is a finding, not an explanation. The confusion matrix for score was more interesting. The lowest level, "not urgent", was never predicted in all 300 emails, even though 77 emails had it as the gold label.

At least three explanations fit that. The ordinal head could be degenerate. The model could be reading position rather than content. Or this particular Japanese phrase could be the problem.

To separate them, I re-ran the same 300 emails under five option schemas: original order, reversed, reworded (low / medium / high), reworded and reversed, and four levels. In all five, the first-listed option was chosen 0 or 1 time out of 300.

The cleanest pair was the original order against the reversed one. "Not urgent" was chosen 0 times when listed first and 250 times when listed last. The emails were the same, the words were the same, and only the slot had changed.

Making it hard to explain away

Before reporting, I tried to list the replies a busy maintainer could reasonably give, and to answer each one with data before he had to ask.

"Your harness is wrong." I re-ran the benchmark using only the two lines from the README, laya.load() and agent.predict(), with no wrapper of mine in between. Then I compared the results to my saved probabilities. The maximum difference was 0.0 after the probability floor I apply, and 2.2e-16 on the raw values.

"It's a Japanese problem." I built 290 English emails with the same label weights, the same generator and the same rejection rules, and ran the same five conditions. The multilingual checkpoint went 0 for 290 in every one of them. English is one of its training languages, so "it can't read the input" doesn't explain this.

First-listed option chosen, per condition (original / reversed / reworded / reworded reversed / four levels):

checkpoint	Japanese (n=300)	English (n=290)
`laya-multilingual`	0, 0, 1, 1, 0	0, 0, 0, 0, 0
`laya` (English)	13, 56, 8, 1, 110	65, 74, 0, 5, 4

The English checkpoint picks the first slot in most conditions, on the same data through the same harness. So the task isn't the cause, and neither is the harness. It comes down to one checkpoint. I opened #131 with a title that said exactly that.

(A correction to myself: in the issue I summarised the English checkpoint as picking the first option "22–26% of the time". That holds for the two original orderings. It is 0 in one condition and under 2% in two others. The table above is the honest version.)

"Is it the position or the word?" Within any single fixed ordering, "the model rejects slot 1" and "the model rejects whichever word sits in slot 1" predict the same table. Someone on X asked whether the effect survives shuffling. Twenty-two minutes later I had condition F: the option order is shuffled per item, with a fixed seed. The results:

The first slot was chosen 0 times out of 300.
Choices by slot were [0, 149, 151].
Choices by label were 75 / 93 / 132.
Each label had sat in the first slot 90 / 109 / 101 times.

So every label lost exactly the picks it had while it was in slot 1. Slots 2 and 3 split evenly, so this isn't a preference for the last slot either. One run was enough to separate position from label.

Someone else went deeper

What happened next was the best part, and it wasn't my work. AlKor13, who is not a maintainer, read the raw marker logits straight off the forward pass, before temperature and softmax.

On the identical code path, swapping only the checkpoint made the hole appear or disappear. Then he ran the control I wish I'd thought of: three identical options, so the inputs differ only by position. The English checkpoint came out essentially flat. The multilingual one came out sharply position-dependent. That settled it: the cause is in the weights, not the code.

He went one step further. Score options are rendered with a level N: prefix, and dropping that prefix made the slot-0 suppression vanish from the raw logits. But he stated the limit of his own result. Without the prefix the input is off-distribution, so the recovery might just be the learned prior getting scrambled. Settling that would take labelled data.

Testing someone else's hypothesis, and my own

I had the labelled data, so I ran three renderings on both benchmarks: A is the shipped level N: rendering, C drops the prefix, and D uses word ordinals. My harness reproduced agent.predict() to within 4.6e-5 under condition A. I compared the renderings with paired McNemar tests on identical items:

bench	change	score accuracy	items that flipped correctness	p
ja	A → C	0.447 → 0.513	56.7%	0.145
ja	A → D	0.447 → 0.570	38.3%	0.0007
en	A → C	0.266 → 0.428	56.9%	0.0003
en	A → D	0.266 → 0.293	19.3%	0.350

Two of the four comparisons are significant, and they are different conditions in different languages. D wins in Japanese and C wins in English, and each is null in the other language. The claim this supports is "the shipped rendering isn't the best one for this checkpoint". It does not support "drop the prefix and it's fixed".

I'll admit the accuracy column alone had me halfway to writing the second claim. The paired test is what stopped me, and I said so in the thread.

The column that matters is the flip rate. Under C, 57% of items change correctness, while headline accuracy moves by 6.7 to 16.2 points. A cosmetic change to the option string nearly re-rolls the prediction. That is an instability, not a mitigation. As a control, removing the prefix makes the English checkpoint worse on the English set (0.583 → 0.500). So level N: isn't harmful in general. The multilingual checkpoint has learned something wrong about that specific pattern.

The maintainer's call

About fifteen hours after I opened the issue, the maintainer, NandhaKishorM, replied to both of us. He called the thread "a model of how to run a bug down" and made three decisions:

The bug is in the weights, not the code.
He would not ship a new default rendering. He cited the 57% flip rate: changing the rendering would swap one instability for another.
The real fix is in training. Ordinal options need the same position balancing that choice options already get.

The next morning, v0.3.7 shipped with a line in the README's limits section. It says laya-multilingual has a position bias on score questions, that English score questions should go to the English checkpoint, and that users should validate score outputs on their own data for other languages. The issue stays open until a retrained checkpoint lands.

I offered to run the same A–F and A/C/D before/after on that checkpoint. He said he'd ping me when it's ready.

Handing over a tool

That offer had a weak point: it depended on me. My benchmarks are CC BY 4.0, so anyone can rerun them. But the question the retrain has to answer doesn't need labels at all: did the slot prior go away? So I wrote PR #259:

python research/eval/presentation_checks.py --model <retrained checkpoint> [--subfolder multilingual]

It uses ten short English support messages that are fixed in the file, with no dataset and no labels. It runs two checks:

score_slot0_identical is AlKor13's identical-option control turned into a gate. It measures slot 0's raw logit minus the slot mean, over K = 3, 4 and 5 identical levels, and requires the result to be at least −0.20.
score_first_slot_permuted puts the three real levels in all six orders for each message, so every level sits in every slot exactly twice. A model whose answer doesn't depend on order picks the first slot in exactly 1/3 of decisions. The gate is at least 0.15.

Results on the checkpoints that ship today:

checkpoint	slot-0 logit (gate ≥ −0.20)	first-slot rate (gate ≥ 0.15)	verdict
`laya` (English)	+0.664	0.217	PASS
`laya-multilingual`	−0.492	0.017	FAIL

I fixed the threshold conditions before the real 10-message run, and I didn't move them afterwards. There were two conditions. First, in every leave-one-out subset, multilingual must fail and English must pass. Second, each checkpoint's worst leave-one-out value must clear its gate by at least 0.10 logit (slot 0) or 0.05 (first-slot rate). The tightest margin turned out to be the English first-slot rate, at 0.054.

The gate is one-sided on purpose. With identical options, the English checkpoint isn't flat either. It leans toward early slots, and the lean grows with K. A two-sided "no position effect" gate would fail the checkpoint everyone agrees is fine. So the gate asks only the narrower question #131 is about: is slot 0 suppressed?

The harness checks itself before it gives a verdict. It first compares its logit path against Agent.system_one on every message. On the shipped checkpoints, the maximum |Δp| was 4.98e-5, which is just the API's 4-decimal rounding. If the difference exceeds 1e-3, the script exits with code 2 and reports no verdict. Exit 0 means pass, 1 means a check failed, and 2 means the harness disagrees with the package. There are 69 offline tests using scripted logits. For example, a scripted slot-0 hole fails both checks, and an order-invariant model scores exactly 1/3.

The PR also lists its own limits. Passing is not accuracy. The check covers English only, score only and ten short messages. And the thresholds were set on CPU with fp32.

Before opening the PR

I spent more time reading the repo than writing the script. Tests there are plain scripts, not pytest. CI doesn't load model checkpoints. Research code lives under research/. A new test has to be registered in both ci.yml and release.yml, and an open PR, #184, hadn't yet settled how research tests get wired in. So I didn't wire mine in. I said so in the PR and offered to do it once #184 lands. Nothing under laya/ changed, and I added no new dependencies.

The same afternoon, another contributor, Rukafuu, had proposed a general metamorphic robustness framework in #244, and the maintainer had invited them to start with option permutation. That was their work to do. I kept my check specific to #131 and noted that it could move into their framework once it exists. Their PR, #269, has since been merged.

259 was merged about three hours after I opened it. The maintainer said a label-free check answering one question was "exactly what #131 needs". He also said that separating a failed check from a harness disagreement in the exit codes made it "easy to trust". That second point is the one I'm proudest of, because it's the part nobody asked for.

On using Claude Code

I used Claude Code throughout: for the harness, the offline tests and drafting the PR text. The commit carries a Co-Authored-By line. The decisions about what to measure, what to claim, what to take back and where to stop were mine, and so are any mistakes.

What I'd tell myself next time

Measure before you report, and check your harness against the official package before you measure anything that matters. "Your harness is wrong" is the cheapest possible dismissal. Rule it out first.
Answer the obvious objections before anyone raises them. Is it the language? Run it in English. The harness? Check parity. The word? Shuffle per item. Each of those came back as one line in the thread instead of a round-trip.
Take back your own overclaims, out loud. McNemar turned my "the prefix is the problem" into "the rendering is unstable". The weaker claim is the one the maintainer could act on.
Credit precisely. The most important control in this story came from AlKor13. My PR is his idea turned into a gate, and the PR says so.
Don't take work someone else was just handed. #244 was Rukafuu's. Keeping my check narrow cost nothing and avoided stepping on them.
Don't claim the fix. I found it, reported it, and merged the tool that will say whether the retrain worked. That's the whole claim.

Where it stands

131 is open, and the retrain is pending. When it lands, `presentation_checks.py` gives the first answer, and I'll run the labelled A–F and A/C/D before/after on top of it.

What I posted the day it merged:

// Detect dark theme var iframe = document.getElementById('tweet-2102969840386687112-803'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2102969840386687112&theme=dark" }

Repo: https://github.com/NandhaKishorM/laya
Issue #131: https://github.com/NandhaKishorM/laya/issues/131
PR #259: https://github.com/NandhaKishorM/laya/pull/259
Related: #171, #244
My data and repro scripts: baseline_ja.md, baseline_en.md

Claude Opus 5.5 vs GPT-6 Astra vs GPT-6 Sol: the effort knob matters more than the model

GeneLab_999 — Wed, 23 Sep 2026 11:27:37 +0000

On September 22, 2026, Anthropic shipped Claude Opus 5.5 and OpenAI shipped GPT-6 Sol on the same day. OpenAI's flagship, GPT-6 Astra, had landed three weeks earlier.

If you tried to pick one from the launch posts, you probably noticed the problem: nobody compared against the right opponent. Anthropic's table compares Opus 5.5 with GPT-6 Astra and the previous GPT-5.6 Sol. OpenAI's Sol post compares with Claude Opus 5 and Fable 5.1, not Opus 5.5. Every chart is technically true and none of them answers "which one should I call from my code today?"

So I went to Artificial Analysis (AA), which ran all three through the same ten evaluations at every reasoning-effort level, and did the cost math.

TL;DR

Raw intelligence: Opus 5.5 (max) scores 58 on AA's Intelligence Index. Astra (max) scores 53, Sol (max) 48.
List price: Sol is cheapest per token, Astra is 2.5x Opus 5.5.
Per-task cost flips the ranking: at max effort Opus 5.5 emits ~4x more output tokens than Astra, so Opus max ($5.98/task) costs more than Astra max ($3.26/task).
The useful finding: below ~$1.30 per task, Sol and Astra trade the lead. Above ~$1.30, Opus 5.5 is on top at every budget. Opus 5.5 at high ($1.82) outscores Astra at max ($3.26).
The effort setting moves your bill more than the model choice does.

1. The price sheet (and its fine print)

Per 1M tokens, standard tier, as of September 23, 2026:

	GPT-6 Sol	GPT-6 Astra	Claude Opus 5.5
Input	$2.00	$10.00	$4.00
Output	$10.00	$50.00	$20.00
Cache read	$0.20	$1.00	$0.20
Cache write	$2.50	$12.50	$5.00 (5 min) / $8.00 (1 h)
Prompt > 272K input tokens	2x input & cache, 1.5x output, whole request	same as Sol	no surcharge up to 1M

Two things developers tend to miss:

Sol and Opus 5.5 have the same cache-read price. Anthropic says cache reads make up the majority of agentic and coding costs. The more your agent re-reads a cached prefix, the less the 2x headline gap matters.
OpenAI's long-context surcharge applies to the entire request, not just the tokens past 272K. For a cache-heavy, long-context agent, Opus 5.5 can end up cheaper than Sol per request.

Also: tokenizers differ between vendors (Anthropic itself notes its newer tokenizer produces roughly 30% more tokens for the same text than its older one). Per-token prices are not a common currency. Compare cost per task instead.

2. Tokens per task is the hidden multiplier

At max effort on AA's suite, Opus 5.5 emits ~119k output tokens per task. Astra emits ~27k, Sol ~31k. That's why Astra, at 2.5x Opus's per-token price, still comes out cheaper per task at the top settings.

Here's the full grid from AA (Intelligence Index score / estimated USD per task):

effort	GPT-6 Sol	GPT-6 Astra	Claude Opus 5.5
low	34 / $0.13	46 / $0.82	42 / $0.55
medium	40 / $0.25	50 / $1.54	51 / $1.34
high	43 / $0.37	51 / $1.73	54 / $1.82
xhigh	44 / $0.53	52 / $2.31	56 / $3.46
max	48 / $1.06	53 / $3.26	58 / $5.98

Note that Opus 5.5 is cheaper than Astra at low and medium, roughly equal at high, and more expensive at xhigh and max.

3. Buy intelligence by the dollar

The question I actually care about: for a given per-task budget, which (model, effort) pair gives the highest score? That's a tiny script:

# Artificial Analysis Intelligence Index v4.3.2 (Sept 22, 2026): (score, est. USD per task)
AA = {
    ("gpt-6-sol", "low"): (34, 0.13),     ("gpt-6-sol", "medium"): (40, 0.25),
    ("gpt-6-sol", "high"): (43, 0.37),    ("gpt-6-sol", "xhigh"): (44, 0.53),
    ("gpt-6-sol", "max"): (48, 1.06),
    ("gpt-6-astra", "low"): (46, 0.82),   ("gpt-6-astra", "medium"): (50, 1.54),
    ("gpt-6-astra", "high"): (51, 1.73),  ("gpt-6-astra", "xhigh"): (52, 2.31),
    ("gpt-6-astra", "max"): (53, 3.26),
    ("claude-opus-5-5", "low"): (42, 0.55),    ("claude-opus-5-5", "medium"): (51, 1.34),
    ("claude-opus-5-5", "high"): (54, 1.82),   ("claude-opus-5-5", "xhigh"): (56, 3.46),
    ("claude-opus-5-5", "max"): (58, 5.98),
}

def best_under(budget_usd: float):
    """Highest-scoring (model, effort) whose estimated cost fits the per-task budget."""
    affordable = [(score, -cost, cfg) for cfg, (score, cost) in AA.items() if cost <= budget_usd]
    if not affordable:
        return None
    score, neg_cost, cfg = max(affordable)  # ties go to the cheaper config
    return cfg, score, -neg_cost

for budget in (0.25, 0.60, 0.90, 1.10, 1.40, 2.00, 4.00, 6.00):
    print(f"${budget:.2f} ->", best_under(budget))

Output:

$0.25 -> (('gpt-6-sol', 'medium'), 40, 0.25)
$0.60 -> (('gpt-6-sol', 'xhigh'), 44, 0.53)
$0.90 -> (('gpt-6-astra', 'low'), 46, 0.82)
$1.10 -> (('gpt-6-sol', 'max'), 48, 1.06)
$1.40 -> (('claude-opus-5-5', 'medium'), 51, 1.34)
$2.00 -> (('claude-opus-5-5', 'high'), 54, 1.82)
$4.00 -> (('claude-opus-5-5', 'xhigh'), 56, 3.46)
$6.00 -> (('claude-opus-5-5', 'max'), 58, 5.98)

Plotted as "best score you can buy at each budget":

Under ~$1.30/task: Sol owns most of the range. Astra low wins a narrow band around $0.82–$1.05.
Over ~$1.30/task: Opus 5.5 leads everywhere (it ties Astra once, briefly, at score 51).
Sol tops out at 48 and Astra at 53. Past those points, paying more buys nothing.

At 10,000 tasks a month, this stops being academic:

Opus 5.5 high scores one point above Astra max and costs $14,400/month less in this illustration. Also note the gap between Opus 5.5 medium and max: about 4.5x the cost for 7 more points.

4. Benchmarks have referees

Anthropic's launch table shows Opus 5.5 at 66.4% on Terminal-Bench 4.0 vs Astra at 57.9%. The footnotes matter: Opus ran at xhigh in Anthropic's setup, and Astra's number is OpenAI's own figure at high.

When AA ran both in the same harness:

59.6% vs 59.6%. A tie. This is the same lesson as Astra's ARC-AGI-3 score, which ARC Prize reported as 99.9% with a provider-specific harness and 62.7% with the standard one. Vendor tables are drawn under each vendor's best conditions. Read the footnotes, and prefer third-party runs that hold the harness constant.

5. Where each model actually wins

From AA at max effort:

	GPT-6 Sol	GPT-6 Astra	Claude Opus 5.5
Terminal-Bench 4.0	43.9%	59.6%	59.6%
AutomationBench-AA	61.6%	69%	69.5%
GDP.pdf (professional documents)	24.8%	31%	26.2%
Hallucination rate (lower is better)	60.1%	51%	58.6%
GDPval-AA (44 occupations, Elo)	1487	1542	1846

Opus 5.5 is far ahead on knowledge work (GDPval-AA is a ~300 Elo lead over Astra). AA's industry sub-indexes agree: finance, legal, and economics all favor Opus.
Astra leads on document reasoning and hallucinates least. In the vendors' own tables it also leads on Terminal-Bench Science (64.6% vs 58.7%, per Anthropic's table).
Astra has several claims Opus 5.5 simply hasn't been measured on: FrontierMath Tier 4 (97.6%), ScreenSpot-Pro (92.7%), MRCR v2 at 512K–1M (96.3%), ExploitBench (100%). No number means no verdict either way.
Sol is the fast, cheap one. It isn't built to win this table.

6. Latency

AA, all at medium effort:

	GPT-6 Sol	GPT-6 Astra	Claude Opus 5.5
Time to first answer token	~2.0 s	~6.2 s	~24.4 s
Output speed	~114 tok/s	~48 tok/s	~75 tok/s
Time per task	~60 s	~198 s	~206 s

For a user-facing chat UI, Sol at medium is in a different league. Crank Astra to max and you're waiting over five minutes for the first answer token.

7. Integration gotchas

Opus 5.5 can't run with thinking disabled. Effort is your only depth control. Sol offers a none reasoning setting (though AA found Sol's non-reasoning mode scored lower and cost more per task than low).
Opus 5.5 routes some requests to other models. When its safeguards trigger, most cybersecurity tasks are handled by Claude Opus 4.8. AA's numbers were measured with this default fallback on.
Astra is rated "Critical" for cyber capability under OpenAI's Preparedness Framework. It currently refuses advanced tasks like proof-of-concept exploits, and OpenAI's misalignment monitoring can stop a task outright in the API.
Mind the 272K line on OpenAI models. A prompt at 280K tokens is billed very differently from one at 270K.

8. A routing recipe you can steal

Default to Opus 5.5 medium for knowledge work and hard agent tasks. It beats Sol's best score and matches Astra high, for less.
Escalate to Opus 5.5 high only when a task fails your acceptance check, rather than starting at max.
Send high-volume, well-specified work to Sol (classification, extraction, first-line support), especially if latency matters.
Keep Astra in the pool for document-heavy reasoning, low-hallucination requirements, science and GUI-heavy work, and verify on your own tasks.

Then measure the only metric that matters, cost per shipped output, including human fix time:

from dataclasses import dataclass

@dataclass(frozen=True)
class Config:
    name: str
    cost_per_attempt: float  # USD, measured on YOUR tasks
    accept_rate: float       # share of outputs you actually ship
    fix_minutes: float       # human repair time per shipped output

def cost_per_accepted(cfg: Config, hourly_rate: float = 60.0) -> float:
    model_cost = cfg.cost_per_attempt / cfg.accept_rate
    human_cost = cfg.fix_minutes / 60 * hourly_rate
    return model_cost + human_cost

# Hypothetical numbers, just to show the shape of the math
for cfg in (
    Config("cheap config", cost_per_attempt=0.25, accept_rate=0.60, fix_minutes=12),
    Config("pricier config", cost_per_attempt=1.34, accept_rate=0.85, fix_minutes=4),
):
    print(f"{cfg.name:>15}: ${cost_per_accepted(cfg):.2f} per shipped output")

   cheap config: $12.42 per shipped output
 pricier config: $5.58 per shipped output

With any realistic hourly rate, human review time dominates the model bill. That is exactly why the "cheapest model" and the "cheapest workflow" are often different things.

Caveats

AA's tasks are not your tasks. Treat the scores as a map, not a forecast.
Costs are AA's estimates, including typical cache-hit rates.
A few per-benchmark values above are rounded integers as AA published them.
AA re-based its index since Astra launched (v4.1.1 → v4.3.2). Scores from different versions aren't comparable.

Sources

Anthropic, Introducing Claude Opus 5.5 and Pricing
OpenAI, Introducing GPT-6 Sol and Luna, GPT-6 Astra, GPT-6 Sol model page, GPT-6 Astra model page
Artificial Analysis, Opus 5.5 vs Astra, Sol vs Opus 5.5, Benchmarking GPT-6 Astra, Opus 5.5 article, Sol and Luna article

Disclosure: I researched and drafted this with help from Claude, which makes Anthropic a party to this comparison. Every figure comes from Artificial Analysis or the vendors' own pages, linked above.

Qiita (Japanese): https://qiita.com/GeneLab_999
Zenn (Japanese): https://zenn.dev/genelab_999
YouTube: https://www.youtube.com/@geneLab_999
X: https://x.com/geneLab_999

I built a Jev-style decision model for Japanese in two days. Claude Code A/B-tested my architecture and killed it.

GeneLab_999 — Tue, 22 Sep 2026 06:35:24 +0000

TL;DR

TypeSafe's Jev introduced "System One" models: no text generation, you send a state and typed questions, you get typed answers with calibrated probabilities in one forward pass. Open reproductions followed within weeks. The most complete one is Laya (ModernBERT-large, 421M).
I benchmarked laya-multilingual on 300 Japanese business emails. choice works. score (ordinal) and bool are below the majority-class baseline.
The score failure is a positional bias: the first-listed option is chosen 0 to 1 times out of 300, under five different orderings and wordings. It reproduces in English (0/290). The English laya checkpoint does not have it. Filed as NandhaKishorM/laya#131.
I built a Japanese one, sokudan-ja-310m, on ModernBERT-Ja. On the same 300 items with unseen schemas it beats laya-multilingual on all three primitives, three seeds. Model, code, benchmark and training-data recipe are public.
Day 1 produced a model that ignored the input text. Day 2 fixed the data and it still failed. The actual cause was my architecture. Claude Code, which I had handed the spec to, ran the A/B that proved it.

Model: https://huggingface.co/GeneLab/sokudan-ja-310m
Code: https://github.com/hiroki-abe-58/sokudan

What a System One model is

If you have ever asked an LLM "which department should handle this ticket, answer with one word" and then written a regex to parse the reply, you know the problem. You are using a text generator as a classifier and paying for it in latency, cost and parse failures.

A System One model skips the generation. Three question types:

type	asks	returns
choice	pick one of N options	choice, per-option probabilities, confidence
score	rate on an ordinal rubric	score, distribution, confidence
bool (Jev calls it noul)	is this statement true	P(true)

Multiple questions go in one request and are evaluated against the same state independently. Laya implements this as an encoder with one [MASK] token per option; the hidden state at each mask is scored and softmaxed within the question. Because the answer space is defined at request time, new schemas need no retraining. On an RTX 5090 it runs at 22 ms per item.

The benchmark

I generated 300 Japanese business emails with a local LLM (Qwen3 30B-A3B), label-conditioned: the prompt says "write a slightly irritated billing inquiry that hints at cancellation", so the generation conditions are the gold labels. No annotation. Three questions per document: 4-way department routing (choice), 3-level urgency (score), "hints at churn" (bool).

Baselines, all on the same 300 items with the same scoring code: laya-multilingual, laya (English checkpoint fed Japanese), majority class, random, and the generating LLM itself as a classifier (an upper bound, not a fair comparison, since it wrote the data).

I did not benchmark Jev itself. TypeSafe's Master Customer Agreement (2.3(b)) prohibits using the service or its output to develop a similar product, and this is one.

Results for laya-multilingual

	choice acc	score RPS (lower is better)	score acc	bool acc	bool AUROC
laya-multilingual	0.747	0.232	0.443	0.543	0.523
majority class	0.380	0.197	0.460	0.703	—
random	0.253	0.201	0.403	0.513	—

choice is fine: twice the majority baseline, three times random. If you need multi-class routing in Japanese, this checkpoint works today.

score is worse than always predicting the majority level. The confusion matrix explains why:

gold \ predicted	soon	work is blocked
not urgent (77)	58	19
soon (138)	78	60
work is blocked (85)	30	55

The lowest level is never predicted. My first guesses were "weak on Japanese negation" or "ordinal training is broken". Five more conditions ruled both out:

condition	options (in listed order)	first option chosen	argmax counts
A original	not urgent / soon / blocked	0 / 300	[0, 167, 133]
B reversed	blocked / soon / not urgent	0 / 300	[0, 50, 250]
C relabeled	low / mid / high	1 / 300	[1, 291, 8]
D relabeled, reversed	high / mid / low	1 / 300	[1, 0, 299]
E four levels	not at all / not urgent / soon / blocked	0 / 300	[0, 78, 37, 185]

Compare B and D. The last-listed option means "not urgent" in one and "high" in the other. Both attract most of the mass (250 and 299). The model is answering by slot, not by label.

Then I ran the same five conditions on 290 English emails, because English is a training language for this checkpoint and "it can't read the input" should not apply:

model	Japanese (n=300)	English (n=290)
laya-multilingual	0, 0, 1, 1, 0	0, 0, 0, 0, 0
laya (English checkpoint)	13, 56, 8, 1, 110	65, 74, 0, 5, 4

Still zero. In English, laya-multilingual also gets score RPS 0.340 (random is 0.197) and bool AUROC 0.355, which is below 0.5, meaning the ranking is inverted. The English checkpoint picks the first option 22 to 26 percent of the time under the same conditions. This is not a language gap; it is something specific to the multilingual checkpoint.

Practical consequence: ordinal rubrics almost always list the lowest level first. On this checkpoint, score will almost never return the lowest level regardless of input.

I verified my harness against the official laya package with laya.load() / agent.predict() straight from the README; probabilities match to float precision. Repro: docs/baseline_en.md.

Temperature scaling, for the record, fixes ECE (0.148 to 0.087) and changes no argmax at all. As expected.

So I built one

sokudan-ja-310m: sbintuitions/modernbert-ja-310m backbone (MIT), full fine-tune, one <mask> per option, softmax within the question. Two design choices worth mentioning:

score is not a multi-class softmax. It uses a cumulative-link parameterization that keeps the CDF monotone for any number of levels K, decided at request time: each level's marker contributes a non-negative increment via softplus, and P(y > k) is a sigmoid over the running sum. K=2 and K=7 share the same head.
All training data is gold-labeled synthetic data from the same label-conditioned generation. No Laya or Jev outputs anywhere.

Results on the same 300 items, three seeds, mean ± SD. The benchmark's three schemas were never in training:

	choice acc	score RPS	score acc	bool acc	bool AUROC
sokudan-ja-310m	0.847 ± 0.009	0.090 ± 0.023	0.763 ± 0.088	0.788 ± 0.010	0.789 ± 0.043
laya-multilingual	0.747	0.232	0.443	0.543	0.523
majority class	0.380	0.197	0.460	0.703	—

Seed variance on score accuracy is large (0.663 / 0.800 / 0.827). The distributed weights are seed 0, which is the worst of the three on that metric and the best on bool AUROC. Seed 0 was designated before any of the three finished training. Seeds 1 and 2 are revisions in the same repo.

Under the five position-bias conditions, sokudan picks the first option 45 to 98 times out of 300 (i.e. normally), across all seeds.

The part I actually want to write about

Day 1 ended with a model whose bool AUROC on the benchmark was 0.513. Diagnostics showed something worse than "weak": replacing the input text with an empty string improved bool accuracy (0.623 to 0.703). The model was reading the question and emitting a prior. My training data's bool questions were all mechanical derivations ("does this document belong to category X", "is urgency at least level k") and none required reading for intent. "Hints at churn" is an intent question. You cannot transfer a skill you never trained.

Day 2 morning: Claude Code and I redesigned the data. Twenty-four attributes baked into the generation prompt as conditions, split into three tiers: surface (mentions a deadline), explicit attitude (states dissatisfaction), implicit intent (implies it is about to decline). Five attributes held out entirely so we had an unseen-schema validation set that was not the benchmark. Claude Code caught two things in the verification pass that I would not have: implicit-intent attributes imply each other, so independent 50/50 sampling creates false labels that are actually true; and discarding verifier disagreements removes hard cases asymmetrically, teaching "always yes". Both fixed.

Day 2 afternoon: trained on the new data. Held-out bool AUROC 0.509. Every held-out attribute at chance, including "ends with a question mark". And choice and score were worse than Day 1 despite 2.6 times the data.

Here is the diagnostic that mattered. On attributes the model had trained on, evaluated on unseen documents, only 4 of 14 were above 0.7, and the top two were "contains numbers" and "mentions a deadline". Lexical. Every attitude and intent attribute sat at 0.5 even though the model had seen thousands of examples of each. Before asking why it did not transfer, we had to ask why it had not learned.

My architecture encoded the state and the question separately and merged them in a two-layer cross-attention head. The selling point was that the state is encoded once per request and broadcast to all questions, so latency does not grow with question count. I had rejected Laya's joint layout (question and state in one sequence) on the grounds that ModernBERT has global attention only every third layer and a 128-token local window, so a question block placed far from the state could not see it.

Claude Code proposed the A/B and ran it: same data, same mix, same epochs, same learning rate, the only change being joint encoding with no cross-attention head.

	cross-attention (mine)	joint (Laya's layout)
held-out bool AUROC, unseen schema	0.506	0.872
implicit-intent tier	0.486	0.786
explicit-attitude tier	—	0.995
trained attributes on unseen docs	4 of 14 above 0.7	14 of 14 between 0.96 and 1.00
seconds per epoch	464	330

Ablating the state (empty or shuffled) drops joint to 0.485 / 0.504. It is reading the text.

My local-attention argument was correct and irrelevant. The mean sequence length in this data is 160 tokens. The "question block at position 3000" I was worried about never occurs. Meanwhile, the only held-out attribute joint failed was "ends with a question mark" (0.688), which is a positional question and exactly where a 128-token window hurts. The concern was real; I had aimed it at the wrong place.

I retracted the design, the latency claim, and the paragraph in the architecture doc that explained why joint would not work. v0.1 ships joint. Claude Code wrote the retraction section with the A/B numbers and a note on where the original argument's premise failed.

Two things I put in the spec that earned their place: never fabricate a number, write TBD (no estimate ever leaked into a doc), and stop at the gate (three sub-majority checkpoints were stopped before publication). One thing I did not put in the spec: permission to refute the spec's author. It did that on its own.

Limits

On the benchmark, bool under-predicts true (mean P(true) 0.125 vs a gold rate of 0.297). AUROC 0.789, so the ranking works; set your threshold to your prior. Temperature does not shift this.
Calibration helps choice and bool ECE and hurts score RPS (0.090 to 0.149). Default is uncalibrated; temperatures.json is included; refit on your own validation set.
Long inputs: held-out bool AUROC drops from 0.904 on short documents to 0.730 at 400 to 800 tokens. Local attention.
Latency scales with question count under joint encoding. usage.backbone_passes in the response tells you how many.
Single benchmark, 300 items, all synthetic from one LLM. Real-data validation is next.

🛠️ I Built a One-Click ComfyUI Setup for RTX 5090 on Windows — No WSL2, No Docker

GeneLab_999 — Mon, 02 Mar 2026 15:18:28 +0000

I bought an RTX 5090. 32GB VRAM. The most powerful consumer GPU on the planet.

Then I tried to run ComfyUI on Windows. It broke immediately.

RuntimeError: sm_120 is not compatible

Three days later, I had a fully working solution. I packaged it and open-sourced it:

ComfyUI-Win-Blackwell

Here's the whole story.

Why RTX 50-series Breaks Everything

NVIDIA's Blackwell architecture (RTX 5090/5080/5070) uses a new compute capability code called sm_120. The problem? PyTorch's stable release doesn't include kernels for it.

This means:

pip install torch → doesn't work on Blackwell
You need PyTorch nightly with CUDA 13.0 (cu130)
But then xformers (the standard ComfyUI speed boost) forces PyTorch back to stable
And custom nodes silently pull stable PyTorch through their dependencies

It's a dependency trap. Every fix creates a new problem.

The 5 Rules I Discovered

After 3 days of trial and error, I distilled everything into 5 rules. Break any one of them and your environment dies.

Rule 1: Use PyTorch nightly cu130 — stable doesn't have sm_120 kernels.

Rule 2: Never install xformers — it force-downgrades PyTorch to stable. This is the trap that got me twice.

Rule 3: Strip torch from every requirements.txt — custom nodes list torch as a dependency, and pip will happily replace your nightly build with stable.

Rule 4: Verify PyTorch after every custom node install — run python -c "import torch; print(torch.__version__)" and check that it still says cu130.

Rule 5: Clear proxy environment variables — system proxies block pip and git silently on Windows.

💡 Pro Tip: Rule 2 was the hardest to figure out. xformers installs successfully, ComfyUI starts fine, and then crashes mid-inference with sm_120 is not compatible. You don't even realize PyTorch was downgraded until you check the version.

What I Built

I automated all 5 rules into a one-click setup:

git clone https://github.com/hiroki-abe-58/ComfyUI-Win-Blackwell.git
cd ComfyUI-Win-Blackwell
# Double-click setup.bat — done in ~20 minutes

What setup.bat handles:

Python 3.13 environment
PyTorch nightly cu130 (not stable, not cu128)
triton-windows + torch.compile (replaces xformers)
ComfyUI core + custom dependencies (with torch stripped out)
28 verified custom nodes
Post-install verification that cu130 is still intact

I also built companion tools:

verify_env.py — Blackwell-specific environment checker (sm_120, cu130, Triton, torch.compile)
fix_windows_compat.py — Converts Linux workflow JSON paths to Windows format, replaces SageAttention with SDPA
update.bat — Updates everything while preserving Blackwell compatibility

What I Verified

I tested 28 custom nodes one by one. Install → check PyTorch version → run test → record result. That was the most tedious part.

I also tested 5 Image-to-Video pipelines on 32GB VRAM:

HunyuanVideo 1.5 I2V (8.3B params, ~16GB) — Smooth. My top recommendation.
Kandinsky 5.0 Lite I2V (2B, ~4GB) — Very smooth. Great for quick tests.
LTX-2 I2V (19B, ~25GB) — Works in FP8. Tight but fine.
LongCat-Video TI2V (13.6B, ~14.5GB) — Works with adjustments.
Kandinsky 5.0 Pro I2V (19B, ~40GB) — Needs CPU offload. Slow.

Why Not Just Use WSL2 or Docker?

The short answer: performance.

Loading safetensors through WSL2's NTFS translation layer is noticeably slower. Docker has the same issue plus additional setup complexity. For a tool like ComfyUI where you're iterating on workflows and loading large models frequently, native Windows file I/O makes a real difference.

Also, most AI artists using ComfyUI on Windows aren't Docker experts. A .bat file they can double-click is the right UX.

Wrapping Up

If you have an RTX 5090/5080/5070 and want to run ComfyUI on Windows without WSL2 or Docker, give it a try:

github.com/hiroki-abe-58/ComfyUI-Win-Blackwell

hiroki-abe-58 / ComfyUI-Win-Blackwell

ComfyUI for GeForce RTX 50-Series (Blackwell)

The first fully documented, Windows-native ComfyUI setup for NVIDIA GeForce RTX 5090/5080/5070 (Blackwell architecture, sm_120) with CUDA 13.0.

Other languages: 日本語 | 中文 | 한국어

What Makes This Special

RTX 50-series GPUs (Blackwell, Compute Capability sm_120) are not supported by PyTorch stable releases as of early 2026. Running ComfyUI on these GPUs requires specific versions and workarounds that are not documented anywhere else in a single, reproducible package.

Technical Highlights

Feature	Details
GPU Architecture	NVIDIA Blackwell (sm_120) -- RTX 5090 / 5080 / 5070
CUDA Version	13.0 (cu130) -- the latest CUDA runtime
PyTorch	Nightly cu130 build (not stable, not cu128)
Python	3.13 (latest)
Triton	triton-windows fork (official Triton is Linux-only)
xformers	Deliberately excluded (causes PyTorch downgrade)
Custom Nodes	28 verified nodes including video & music generation
Platform	Windows Native (no WSL2, no Docker required)

Why This Is Unique

Blackwell + Windows Native +…

View on GitHub

MIT licensed. Stars and PRs welcome — especially if you verify additional custom nodes on Blackwell hardware.

Have you tried running AI tools on RTX 50-series? What was your experience? Let me know in the comments! 👇

If you found this helpful, consider following me for more AI + GPU content!

📝 Japanese version: Qiita / Zenn
🐦 Follow me on X: @geneLab_999
💻 GitHub: hiroki-abe-58

How I Run 6 AI Services Simultaneously on RTX 5090 + WSL2 + Docker (And You Can Too)

GeneLab_999 — Sat, 21 Feb 2026 22:53:44 +0000

TL;DR:

I built a multi-service local AI stack (image gen, video gen, voice synthesis, voice cloning) running on RTX 5090 via WSL2 Docker. The key breakthrough was solving the GPU driver passthrough layer that nobody documented. Here's the architecture, the critical gpu-run function, and everything I learned the hard way.

The Problem Nobody Solved

In August 2025, I bought an RTX 5090. Blackwell architecture. 32GB GDDR7. Compute capability sm_120.

And nobody could make it work with WSL2 + Docker + PyTorch.

The issue wasn't any single component. nvidia-smi worked fine in containers. libcuda.so.1 loaded correctly. But PyTorch kept returning torch.cuda.is_available() = False with a cryptic Error 500: named symbol not found.

I spent roughly 40 hours debugging. Here's what I found, and how I turned it into a production multi-service AI environment.

The Root Cause

The failure point was in the interaction layer between WSL2's driver mounting and Docker's GPU runtime.

When you run --gpus all in a Docker container on WSL2, the NVIDIA Container Toolkit mounts /usr/lib/wsl/lib into the container. This directory contains libcuda.so.1 and friends. For most GPUs, this is enough.

For the RTX 5090, it's not.

The actual driver binaries live in a separate directory: /usr/lib/wsl/drivers/nvmdi.inf_amd64_<hash>. This directory contains the real libcuda.so.1.1, libnvdxgdmal.so.1, libnvidia-ptxjitcompiler.so.1, and other dependencies that the PyTorch CUDA runtime needs to initialize the Blackwell architecture.

Without mounting this directory AND setting LD_LIBRARY_PATH to include it, PyTorch's CUDA initialization hits a dead end -- it finds libcuda.so.1 but can't resolve the sm_120-specific symbols.

The Solution: `gpu-run`

Here's the function that makes everything work:

gpu-run () {
  local D BN
  D=$(ls -d /usr/lib/wsl/drivers/nvmdi.inf_amd64_* | head -n1) || return 1
  BN=$(basename "$D")
  echo "Using driver path: $D"
  docker run --rm --gpus all \
    -v /usr/lib/wsl/lib:/usr/lib/wsl/lib:ro \
    -v "$D":/usr/lib/wsl/drivers/"$BN":ro \
    -e LD_LIBRARY_PATH=/usr/lib/wsl/lib:/usr/lib/wsl/drivers/"$BN" \
    "$@"
}

What this does:

Finds the driver directory dynamically -- the hash suffix changes with driver updates
Mounts both WSL lib paths -- the standard /usr/lib/wsl/lib AND the driver-specific directory
Sets LD_LIBRARY_PATH to prioritize these paths for symbol resolution

Verification:

source gpu-run.sh
gpu-run torch-wsl-cu128 python3 -c "
import torch
print('PyTorch:', torch.__version__)
print('CUDA available:', torch.cuda.is_available())
print('GPU:', torch.cuda.get_device_name(0))
print('VRAM:', torch.cuda.get_device_properties(0).total_mem // 1024**3, 'GB')
"

Output:

Using driver path: /usr/lib/wsl/drivers/nvmdi.inf_amd64_fb80e95fa979ce23
PyTorch: 2.9.0.dev20250812+cu128
CUDA available: True
GPU: NVIDIA GeForce RTX 5090
VRAM: 32 GB

The Dockerfile Template

Every AI service in my stack uses a variation of this base:

FROM nvidia/cuda:12.8.0-devel-ubuntu22.04

ENV TZ=Asia/Tokyo
ENV DEBIAN_FRONTEND=noninteractive
ENV PYTHONUNBUFFERED=1
ENV CUDA_HOME=/usr/local/cuda

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 python3-pip python3-dev git ffmpeg ca-certificates \
    build-essential cmake ninja-build libsndfile1 \
    && rm -rf /var/lib/apt/lists/*

RUN pip3 install --upgrade pip
RUN pip3 install --no-cache-dir numpy==1.26.4

RUN pip3 install --no-cache-dir --pre \
    torch torchvision torchaudio \
    --index-url https://download.pytorch.org/whl/nightly/cu128

RUN python3 -c "import torch; print('PyTorch:', torch.__version__); assert 'cu128' in torch.__version__"

WORKDIR /app

Key decisions:

nvidia/cuda:12.8.0-devel-ubuntu22.04 -- CUDA 12.8 is the minimum for sm_120. Using devel (not runtime) because some AI frameworks compile CUDA extensions at build time.
PyTorch nightly cu128 -- as of early 2026, stable PyTorch still has incomplete Blackwell support. Nightly cu128 is non-negotiable.
numpy pinned to 1.26.4 -- numpy 2.x breaks several AI frameworks that haven't updated their C extensions.
Install torch LAST -- many requirements.txt files include torch. If you install dependencies first, they'll pull in a stable torch that doesn't support sm_120. Always install your carefully selected torch version as the final step.

Docker Compose Architecture

Here's how six AI services coexist in a single compose.yaml:

services:
  comfyui:
    build:
      context: ./apps/comfyui
      dockerfile: Dockerfile
    image: comfyui:wsl-cu12
    profiles: ["comfyui", "all"]
    runtime: nvidia
    environment:
      - LD_LIBRARY_PATH=/usr/lib/wsl/lib:/usr/lib/wsl/drivers/${WSL_DRV_BN}
      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:512
      - CUDA_VISIBLE_DEVICES=0
    volumes:
      - /usr/lib/wsl/lib:/usr/lib/wsl/lib:ro
      - ${WSL_DRV_DIR}:/usr/lib/wsl/drivers/${WSL_DRV_BN}:ro
      - ./data/comfyui-models:/app/models
      - ./shared/models:/shared/models:ro
    ports:
      - "8188:8188"
    ipc: host
    ulimits:
      memlock: -1
      stack: 67108864

  sbv2:
    build:
      context: ./apps/sbv2
      dockerfile: Dockerfile
    image: sbv2:wsl-cu12
    profiles: ["sbv2", "all"]
    runtime: nvidia
    environment:
      - LD_LIBRARY_PATH=/usr/lib/wsl/lib:/usr/lib/wsl/drivers/${WSL_DRV_BN}
      - PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True,max_split_size_mb:512
    volumes:
      - /usr/lib/wsl/lib:/usr/lib/wsl/lib:ro
      - ${WSL_DRV_DIR}:/usr/lib/wsl/drivers/${WSL_DRV_BN}:ro
      - ./data/sbv2-models:/opt/models
    ports:
      - "5000:5000"
    ipc: host
    ulimits:
      memlock: -1
      stack: 67108864

  cosyvoice:
    profiles: ["cosyvoice", "all"]
    ports:
      - "7865:7865"

  rvc:
    profiles: ["rvc", "all"]
    ports:
      - "7866:7866"

  framepack:
    profiles: ["framepack", "all"]
    ports:
      - "7862:7862"

(Each service follows the same WSL driver mount pattern -- I've abbreviated the later ones for readability.)

The .env file is auto-generated:

WSL_DRV_DIR=$(ls -d /usr/lib/wsl/drivers/nvmdi.inf_amd64_* | head -n1)
WSL_DRV_BN=$(basename "$WSL_DRV_DIR")
cat > .env << EOF
WSL_DRV_DIR=$WSL_DRV_DIR
WSL_DRV_BN=$WSL_DRV_BN
EOF

Design Decisions That Saved My Sanity

1. Docker Profiles for Resource Isolation

With 32GB VRAM, you can't run everything simultaneously. Video generation alone can eat 24GB. Docker profiles let me spin up exactly what I need:

docker compose --profile comfyui up -d
docker compose --profile sbv2 --profile cosyvoice up -d
docker compose --profile all up -d

2. Shared Model Directory

AI models are enormous. Flux checkpoints, HunyuanVideo weights, voice models -- easily 200GB+. Instead of duplicating them per container:

~/ai-workspace-correct/
  shared/
    models/           # Cross-service shared models
    hf_cache/         # HuggingFace cache (persistent)
  data/
    comfyui-models/   # Service-specific models
    sbv2-models/
    cosyvoice-models/

Each service mounts shared/models read-only. Service-specific models go in their own data/ directory.

3. Port Allocation Strategy

I carved out port ranges by domain:

Range	Domain	Services
5000-5009	Voice synthesis	Style-BERT-VITS2
7860-7869	Voice/Video AI	FramePack, CosyVoice, RVC
8180-8189	Image AI	ComfyUI

This avoids collisions and makes firewall rules predictable.

4. The torchaudio Trap

This one cost me hours. Several voice synthesis frameworks use torchaudio.info() and torchaudio.load(). The nightly cu128 build of torchaudio has breaking API changes. The fix:

import soundfile as sf

sample_rate = sf.info(wav_path).samplerate
audio_data, sr = sf.read(wav_path)

I patch these at Docker build time with sed:

RUN sed -i 's/import torchaudio/import torchaudio\nimport soundfile as sf/' /opt/app/webui.py && \
    sed -i 's/torchaudio.info(prompt_wav).sample_rate/sf.info(prompt_wav).samplerate/g' /opt/app/webui.py

Lessons Learned (The Hard Way)

1. Never let requirements.txt install torch.
Strip torch, torchvision, torchaudio from every requirements.txt before installing. Then install your nightly cu128 build as the final step. If you don't, pip will happily overwrite your working torch with a stable version that can't see your GPU.

2. Driver updates break the hash.
The nvmdi.inf_amd64_<hash> directory changes when you update NVIDIA drivers. The gpu-run function handles this with dynamic lookup. But if you hardcode the path anywhere, you'll have a bad time.

3. ipc: host is non-negotiable for AI workloads.
Without it, PyTorch's shared memory operations fail silently or with cryptic errors. Always set it.

4. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
This environment variable enables PyTorch's memory-efficient allocation strategy. Without it on 32GB VRAM, you'll hit fragmentation issues on large models that shouldn't theoretically run out of memory.

5. Document everything as if you'll have amnesia tomorrow.
I wrote my setup docs with the goal of "restore everything from scratch in 30 minutes." That document has saved me three times already.

Current Stack (February 2026)

Service	Purpose	Port	Status
ComfyUI	Image generation (Flux, SDXL)	8188	Stable
Style-BERT-VITS2	Japanese TTS voice synthesis	5000	Stable
CosyVoice	Multi-speaker voice synthesis	7865	Stable
RVC	Real-time voice conversion	7866	Stable
FramePack	Video generation (HunyuanVideo)	7862	Stable

All running on:

GPU: RTX 5090 32GB GDDR7
CPU: Intel Core Ultra 9 285K
RAM: 64GB DDR5
OS: Windows 11 Pro + WSL2 Ubuntu 22.04
Container runtime: Docker with NVIDIA Container Toolkit

Is This Still Unique?

As of February 2026, there are published examples of single-service RTX 5090 + Docker setups (vLLM, ComfyUI, basic PyTorch). What I haven't found elsewhere is:

A multi-service Docker Compose stack orchestrating 5+ AI services on Blackwell
The specific WSL2 driver mount solution documented with the nvmdi.inf_amd64_* path
A systematic approach to dependency isolation across services sharing one GPU
Production-grade patterns for model sharing, port management, and environment recovery

If you've done something similar, I'd genuinely love to hear about it. Drop a comment or reach out.

Built with ~40 hours of debugging, 200+ GB of model files, and an unreasonable amount of stubbornness. Based in Tokyo.

#rtx5090 #docker #wsl2 #pytorch #cuda #blackwell #ai #selfhosted

I Built a Voice Cloning GUI That Supports 10 Languages — Here's What I Learned Wrestling with CUDA on Windows published

GeneLab_999 — Sat, 21 Feb 2026 22:05:52 +0000

Have you ever recorded yourself speaking and thought, "I wish I could just type what I want to say and have my own voice read it back"?

That's exactly the rabbit hole I fell down when Alibaba dropped Qwen3-TTS — an open-source TTS model that can clone any voice from just 3 seconds of audio. Ten languages. 97ms latency. Apache 2.0 license. On paper, it was everything I'd ever wanted.

In practice? It assumed Linux. FlashAttention 2 (recommended) doesn't run on Windows. And voice cloning required you to manually transcribe your reference audio — which kind of defeats the purpose of a "quick clone" workflow.

So I did what any developer would do: I forked it.

What I Built

hiroki-abe-58 / Qwen3-TTS-JP

Japanese GUI + Whisper auto-transcription for Qwen3-TTS. RTX 5090 tested.

Qwen3-TTS-JP

A Windows-native fork of Qwen3-TTS with a modern, multilingual Web UI.

The original Qwen3-TTS was developed primarily for Linux environments, and FlashAttention 2 is recommended. However, FlashAttention 2 does not work on Windows. This fork enables direct execution on Windows without WSL2 or Docker, provides a modern Web UI supporting 10 languages, and adds automatic transcription via Whisper.

Mac (Apple Silicon) users: For the best experience on Mac, please use Qwen3-TTS-Mac-GeneLab -- fully optimized for Apple Silicon with MLX + PyTorch dual engine, 8bit/4bit quantization, and 10-language Web UI.

Custom Voice -- Speech synthesis with preset speakers

Voice Design -- Describe voice characteristics to synthesize

Voice Clone -- Clone voice from reference audio

Settings -- GPU / VRAM / Model information

Related Projects

Platform	Repository	Description
Windows	This

…

View on GitHub

Qwen3-TTS-JP started as a personal fix — a Japanese-localized fork with Whisper auto-transcription bolted on. But as people started using it, I realized the same pain points existed for developers everywhere. So I expanded it:

10-language Web UI — Japanese, English, Chinese, Korean, German, French, Russian, Portuguese, Spanish, Italian. The UI auto-detects your browser locale.
Native Windows support — No WSL. No Docker. Just Python + CUDA.
Whisper auto-transcription — Upload 3 seconds of audio, Whisper handles the rest. Pick from 5 model sizes (tiny → large-v3) depending on your speed/accuracy tradeoff.
RTX 5090 (Blackwell) tested — I developed this on a Blackwell GPU, so sm_120 architecture is a first-class citizen.
Mac support — Apple Silicon users get a dedicated fork with MLX + PyTorch dual engine and 4bit/8bit quantization.

The Architecture in 30 Seconds

Qwen3-TTS isn't your typical TTS pipeline. Instead of the usual Text → LM → DiT → Audio cascade, it uses a discrete multi-codebook LM that goes straight from text to audio codes:

Traditional:  Text → Language Model → Intermediate Repr → DiT → Audio
Qwen3-TTS:    Text → Language Model → Audio Codes → Decoder → Audio

This bypasses the information bottleneck that makes most TTS systems sound robotic. The result is eerily human-sounding output — with emotion, prosody, and natural pauses all preserved.

The dual-track streaming architecture means it starts generating audio from the first character of input. That 97ms first-packet latency is real.

Getting It Running (It's Actually Easy Now)

git clone https://github.com/hiroki-abe-58/Qwen3-TTS-JP.git
cd Qwen3-TTS-JP

python -m venv .venv
# Windows
.venv\Scripts\activate
# Linux/Mac
source .venv/bin/activate

pip install -e .
pip install faster-whisper

# RTX 30/40 series
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# RTX 50 series (Blackwell) — needs nightly
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Launch the GUI:

# Voice cloning mode
python -m qwen_tts.cli.demo Qwen/Qwen3-TTS-12Hz-1.7B-Base \
    --ip 127.0.0.1 --port 7860 --no-flash-attn

Open http://127.0.0.1:7860. Done.

What You Can Actually Build With This

Here's where it gets interesting for developers. This isn't just a toy — the Python API is clean enough to integrate into real projects.

Voice Cloning in 5 Lines

from qwen_tts import Qwen3TTSModel
import torch, soundfile as sf

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0", dtype=torch.bfloat16,
)

wavs, sr = model.generate_voice_clone(
    text="This is my cloned voice. It only needed 3 seconds of audio.",
    language="English",
    ref_audio="my_voice.wav",       # 3 seconds is enough
    ref_text="Hello, testing.",      # Whisper can auto-generate this
)
sf.write("output.wav", wavs[0], sr)

Design a Voice From Scratch

No reference audio needed — just describe what you want:

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign",
    device_map="cuda:0", dtype=torch.bfloat16,
)

wavs, sr = model.generate_voice_design(
    text="Welcome back, adventurer. Your quest awaits.",
    language="English",
    instruct="Deep male voice, 45 years old, slight British accent, warm and commanding",
)

Cross-Lingual Cloning

Clone a voice in one language, generate speech in another. The model preserves the speaker's timbre across languages:

wavs, sr = model.generate_voice_clone(
    text="Bonjour, comment allez-vous aujourd'hui?",
    language="French",
    ref_audio="english_speaker.wav",
    ref_text="Hi, this is a test recording.",
)

Practical Use Cases I've Seen

Since releasing this fork, I've seen developers use it for:

Game dev — Generating NPC dialogue dynamically instead of recording thousands of audio files
Podcasting — Creating consistent intro/outro narration
Accessibility — Multilingual audio versions of documentation
Localization — Same voice, 10 languages, zero re-recording
Prototyping — Testing voice UX before hiring voice actors

GPU Compatibility

GPU	VRAM	Recommended Model	Status
RTX 5090	32GB	1.7B	Tested & verified
RTX 4090	24GB	1.7B	Works great
RTX 4070	12GB	0.6B or 1.7B (tight)	Works
RTX 3080	10GB	0.6B	Works
Apple Silicon	16GB+	Via Mac fork	MLX optimized

If you're VRAM-constrained, the 0.6B model is surprisingly capable — and FlashAttention 2 can help on Linux:

pip install flash-attn --no-build-isolation

Things I Learned the Hard Way

A few gotchas from building this that might save you time:

Windows cp932 encoding hell. Japanese Windows defaults to cp932 encoding, which chokes on Unicode output from the model. The fix is wrapping stdout/stderr:

import sys, io
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')

FlashAttention 2 doesn't compile on Windows. The solution is using PyTorch's built-in SDPA (Scaled Dot Product Attention) via --no-flash-attn. Performance hit is minimal for single-user inference.

Blackwell (sm_120) needs nightly PyTorch. As of early 2026, stable PyTorch doesn't support RTX 50-series. Nightly builds with cu128 work, but you'll see warnings about torchao version mismatches. They're cosmetic — ignore them.

SoX is optional. The model prints warnings about missing SoX, but it works fine without it. Don't waste time installing it on Windows.

What's Next

I'm currently exploring:

vLLM integration for production-grade serving
Fine-tuning workflows for custom voice models
Streaming WebSocket API for real-time applications

Try It

If you're working on anything voice-related — games, accessibility, content creation, or just want to mess around with state-of-the-art TTS — give it a spin:

Windows/Linux: Qwen3-TTS-JP
Mac (Apple Silicon): Qwen3-TTS-Mac-GeneLab

Stars are appreciated — they help other developers find the project.

I'm curious: What would you build with 3-second voice cloning? Drop your ideas in the comments — I'd love to hear what use cases I haven't thought of yet.

Ethical Note

Voice cloning is powerful tech. Please use it responsibly — clone only with consent, disclose AI-generated audio, and don't use it for fraud or impersonation. The Apache 2.0 license gives you freedom, but with great power... you know the rest.

ComfyUI-AceMusic: The First Full Implementation of ACE-Step 1.5 Features That "Weren't Yet Supported"

GeneLab_999 — Wed, 04 Feb 2026 15:06:27 +0000

TL;DR

On February 3rd, 2026, the official ComfyUI blog announced ACE-Step 1.5 support with a notable caveat: "Cover, Repaint, and other features aren't yet supported in ComfyUI."

The next day, I released ComfyUI-AceMusic — a complete implementation of all 15 ACE-Step 1.5 features as ComfyUI nodes.

Key highlights:

World-first: Full Cover, Repaint, Edit, Retake, Extend support in ComfyUI
15 nodes covering every ACE-Step 1.5 capability
Modular architecture that eliminates widget ordering issues
Windows + Python 3.13+ compatible using soundfile/scipy instead of problematic torchaudio backends
HeartMuLa interoperability for hybrid AI music workflows

GitHub: github.com/hiroki-abe-58/ComfyUI-AceMusic

The Problem: Official Support Was Incomplete

ACE-Step 1.5 is a game-changer for open-source music generation. It outperforms most commercial alternatives, runs on consumer hardware (4GB VRAM), and generates full songs in under 10 seconds on an RTX 3090.

When ComfyUI announced native support, the community was excited. But there was a catch.

From the official ComfyUI blog (February 3rd, 2026):

"ACE-Step 1.5 has a few more tricks up its sleeve. These aren't yet supported in ComfyUI, but we have no doubt the community will figure it out."

The "tricks" they mentioned? Only the most powerful features of ACE-Step 1.5:

Feature	Description	Official Support
Cover	Transform any song into a different style	❌ Not supported
Repaint	Regenerate specific sections of audio	❌ Not supported
Edit	Change tags/lyrics while preserving melody	❌ Not supported
Retake	Create variations of existing audio	❌ Not supported
Extend	Add new content before/after audio	❌ Not supported

So I built them.

What ComfyUI-AceMusic Offers

Complete Feature Coverage

Node	Function
Model Loader	Downloads and caches ACE-Step 1.5 models
Settings	Configure generation parameters
Generator	Text-to-Music generation
Lyrics Input	Dedicated lyrics input with section markers
Caption Input	Style/genre description input
Cover	Transform existing audio into different styles
Repaint	Regenerate specific time ranges
Retake	Create variations with same settings
Extend	Add content to beginning or end
Edit	Change tags/lyrics, preserve melody (FlowEdit)
Conditioning	Combine parameters into conditioning object
Generator (from Cond)	Generate from conditioning
Load LoRA	Load fine-tuned adapters
Understand	Extract metadata from audio
Create Sample	Generate params from natural language

Comparison with Existing Implementations

Implementation	ACE-Step Version	Cover	Repaint	Edit	Retake	Extend	Win 3.13+
ComfyUI Native	1.5	❌	❌	❌	❌	❌	Untested
billwuhao	1.0	Partial	✅	❌	❌	✅	Untested
ryanontheinside	1.0	❌	✅	❌	❌	✅	Untested
ComfyUI-AceMusic	1.5	✅	✅	✅	✅	✅	✅

Technical Deep Dive

1. Modular Architecture

Previous implementations crammed 30+ parameters into a single node, causing widget ordering issues — a known ComfyUI quirk where input field order can cause unexpected behavior.

ComfyUI-AceMusic separates concerns:

[Model Loader] → Model loading only
[Settings] → Generation parameters only  
[Lyrics Input] → Lyrics entry only
[Caption Input] → Style description only
[Generator] → Generation execution only

This separation:

Eliminates widget ordering bugs
Improves workflow readability
Makes nodes reusable across different workflows
Follows single-responsibility principle

2. Cross-Platform Compatibility

The Problem: torchaudio backends can fail on Windows + Python 3.13+.

The Solution: Use soundfile and scipy instead.

# Problematic approach
import torchaudio
audio, sr = torchaudio.load("file.wav")  # Fails on Windows 3.13+

# ComfyUI-AceMusic approach
import soundfile as sf
audio, sr = sf.read("file.wav")  # Works everywhere

This isn't just a workaround — it's a more robust solution that works across all platforms without requiring specific backend configurations.

3. HeartMuLa Interoperability

The AUDIO type in ComfyUI-AceMusic is compatible with HeartMuLa outputs, enabling hybrid workflows:

[HeartMuLa Generator] → [AceMusic Cover] → [AceMusic Extend] → [Output]

This lets you combine the strengths of different music generation models in a single workflow.

Quick Start

Installation

Via ComfyUI Manager (Recommended):
Search for "ComfyUI-AceMusic" and install.

Manual:

cd ComfyUI/custom_nodes
git clone https://github.com/hiroki-abe-58/ComfyUI-AceMusic.git
cd ComfyUI-AceMusic
pip install -r requirements.txt

# Install ACE-Step 1.5
pip install git+https://github.com/ace-step/ACE-Step.git

Models auto-download from Hugging Face on first use.

Basic Workflow (Text-to-Music)

Add AceMusic Model Loader → set device to cuda
Add AceMusic Settings → configure duration, language, etc.
Add AceMusic Lyrics Input:

   [Verse]
   Walking down the empty street
   Thinking about you and me

   [Chorus]
   We belong together
   Now and forever

Add AceMusic Caption Input: pop, female vocal, energetic
Connect all to AceMusic Generator → Preview Audio

Load the example workflow: workflow/AceMusic_Lyrics_v3.json

Cover Workflow (Style Transfer)

[Load Audio] ──────────────────┐
                               ↓
[Model Loader] → [Settings] → [AceMusic Cover] → [Preview Audio]
                               ↑
[Caption Input] ───────────────┘
"jazz piano trio, smooth, relaxed"

Use cases:

Pop → Jazz arrangement
Rock → Acoustic version
EDM → Orchestral arrangement

Repaint Workflow (Section Regeneration)

[Load Audio] ──────────────────┐
                               ↓
[Model Loader] → [Settings] → [AceMusic Repaint] → [Preview Audio]
                               ↑
[Time Range: 30-45s] ──────────┘

Use cases:

Fix a problematic chorus
Improve the intro
Regenerate specific vocal sections

Performance

Generation Speed

Device	RTF (27 steps)	Time for 1 min audio
RTX 5090	~50x	~1.2s
RTX 4090	34.48x	1.74s
A100	27.27x	2.20s
RTX 3090	12.76x	4.70s
M2 Max	2.27x	26.43s

VRAM Requirements

Mode	VRAM	Notes
Normal	8GB+	Full speed
CPU Offload	~4GB	Slower but works on limited VRAM

Troubleshooting

Error	Cause	Solution
`CUDA out of memory`	Insufficient GPU memory	Enable `cpu_offload` or reduce `duration`
`ModuleNotFoundError: acestep`	ACE-Step not installed	`pip install git+https://github.com/ace-step/ACE-Step.git`
`soundfile not found`	Missing dependency	`pip install soundfile scipy`
`Model download failed`	Network issue	Check Hugging Face access
`torchaudio backend error`	Windows 3.13+ issue	Ensure soundfile is properly installed

Environment Check Script

#!/usr/bin/env python3
"""ComfyUI-AceMusic Environment Checker"""
import sys

def check():
    issues = []

    # Python version
    print(f"Python: {sys.version}")
    if sys.version_info < (3, 10):
        issues.append("Python 3.10+ required")

    # PyTorch + CUDA
    try:
        import torch
        print(f"✅ PyTorch: {torch.__version__}")
        if torch.cuda.is_available():
            print(f"✅ CUDA: {torch.version.cuda}")
            vram = torch.cuda.get_device_properties(0).total_memory / 1e9
            print(f"✅ GPU VRAM: {vram:.1f} GB")
        else:
            issues.append("CUDA not available")
    except ImportError:
        issues.append("PyTorch not installed")

    # ACE-Step
    try:
        import acestep
        print("✅ ACE-Step: installed")
    except ImportError:
        issues.append("ACE-Step not installed")

    # Audio libraries
    try:
        import soundfile
        print("✅ soundfile: installed")
    except ImportError:
        issues.append("soundfile not installed")

    # Results
    print("\n" + "="*50)
    if issues:
        print("❌ Issues found:")
        for issue in issues:
            print(f"  - {issue}")
    else:
        print("✅ Environment OK!")

if __name__ == "__main__":
    check()

Why I Built This

When I saw the official announcement saying "these features aren't yet supported," I knew exactly what needed to be done. The ACE-Step team built an incredible model with Cover, Repaint, Edit, and other powerful features — but without ComfyUI support, most users couldn't access them.

The hardest part was the torchaudio issue. On Windows with Python 3.13+, the audio backends just don't work reliably. The solution was to bypass torchaudio entirely and use soundfile/scipy for all audio I/O. It's a more robust approach that should work on any platform.

The modular architecture came from frustration with existing implementations. Stuffing 30+ parameters into one node isn't just ugly — it causes real bugs. Separating concerns made the nodes more reliable and the workflows more readable.

This is what open source is about. The official team sets the direction, and the community fills in the gaps. I'm proud to contribute to the music generation ecosystem.

License

Apache 2.0

If you find this useful, consider starring the repo. And if you build something cool with it, I'd love to see it!

Run Qwen3-TTS on Windows with RTX 5090: Voice Cloning in 3 Seconds

GeneLab_999 — Sat, 31 Jan 2026 09:37:20 +0000

Run Qwen3-TTS on Windows with RTX 5090: The Complete Guide to Voice Cloning in 3 Seconds

Clone any voice with just 3 seconds of audio — now with native Windows support and the latest Blackwell GPUs

TL;DR

Qwen3-TTS-JP is a fork of Alibaba's Qwen3-TTS that adds:

✅ Native Windows support (no WSL required!)
✅ RTX 5090 / Blackwell GPU tested and optimized
✅ Auto-transcription via Whisper integration
✅ Localized GUI (Japanese, easy to adapt)

hiroki-abe-58 / Qwen3-TTS-JP

Japanese GUI + Whisper auto-transcription for Qwen3-TTS. RTX 5090 tested.

Qwen3-TTS-JP

A Windows-native fork of Qwen3-TTS with a modern, multilingual Web UI.

Mac (Apple Silicon) users: For the best experience on Mac, please use Qwen3-TTS-Mac-GeneLab -- fully optimized for Apple Silicon with MLX + PyTorch dual engine, 8bit/4bit quantization, and 10-language Web UI.

Custom Voice -- Speech synthesis with preset speakers

Voice Design -- Describe voice characteristics to synthesize

Voice Clone -- Clone voice from reference audio

Settings -- GPU / VRAM / Model information

Related Projects

Platform	Repository	Description
Windows	This

…

View on GitHub

The Problem: Getting Qwen3-TTS Running on Windows

When Alibaba released Qwen3-TTS in January 2026, the AI community was amazed: 3 seconds of reference audio is all you need to clone a voice. Ten languages supported, 97ms latency, emotion control — impressive specs on paper.

But there was a catch.

The official repo assumed Linux. CUDA setup was finicky. And if you wanted to use the voice cloning feature, you had to manually transcribe your reference audio — defeating the purpose of a quick workflow.

I'd just upgraded to an RTX 5090 (Blackwell architecture), eager to push local AI to its limits. After days of wrestling with environments, I got it working and decided to package the solution for everyone else.

What Makes This Fork Different?

1. Native Windows Support

No WSL. No Docker (though it's optional). Just Python, CUDA, and you're good to go.

# Clone the repo
git clone https://github.com/hiroki-abe-58/Qwen3-TTS-JP.git
cd Qwen3-TTS-JP

# Create virtual environment
python -m venv .venv
.venv\Scripts\activate

# Install
pip install -e .
pip install faster-whisper

That's it. Works on Windows 10/11 with any CUDA-capable GPU (RTX 30/40/50 series).

2. RTX 5090 (Blackwell) Tested

This fork was developed and tested on an RTX 5090. The latest CUDA 12.x with Blackwell architecture can be tricky — many AI repos break on it. This one doesn't.

GPU	VRAM	Model	Status
RTX 5090	32GB	1.7B	✅ Works
RTX 4090	24GB	1.7B	✅ Works
RTX 3080	10GB	0.6B	✅ Works

3. Whisper Auto-Transcription

The original Qwen3-TTS requires you to provide the transcript of your reference audio. This fork integrates faster-whisper to do it automatically:

Upload 3 seconds of audio
Whisper transcribes it
Qwen3-TTS clones the voice

No manual typing. Choose from 5 Whisper models:

Model	Params	Speed	Accuracy
tiny	39M	⚡⚡⚡⚡⚡	★★
small	244M	⚡⚡⚡	★★★★
large-v3	1.5B	⚡	★★★★★

Quick Start

Launch the GUI

python -m qwen_tts.demo Qwen/Qwen3-TTS-12Hz-1.7B-Base \
    --ip 0.0.0.0 --port 8000

Open http://localhost:8000 in your browser.

Python API

import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

model = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
)

# Clone a voice with 3-second reference
wavs, sr = model.generate_voice_clone(
    text="This is my cloned voice speaking!",
    language="English",
    ref_audio="reference.wav",
    ref_text="Hello, this is a test.",
)
sf.write("output.wav", wavs[0], sr)

Use Cases

Content Creators

Clone your own voice for consistent narration across videos.

Game Developers

Create character voices without expensive voice actors:

model.generate_voice_design(
    text="Hero, your quest awaits!",
    language="English",
    instruct="Deep male voice, 40 years old, British accent"
)

Podcasters

Quick voice-over generation for intros and outros.

Supported Languages

Troubleshooting

Error	Fix
`CUDA out of memory`	Use 0.6B model or add FlashAttention 2
`faster-whisper not found`	`pip install faster-whisper`

Ethical Note

Voice cloning is powerful. Please:

Only clone voices with consent
Don't use for fraud or misinformation
Disclose AI-generated audio

Links

If this helped, please ⭐ the repo!

Questions? Drop a comment below! 👇

DEV Community: GeneLab_999

I didn't fix the bug: contributing to a 20k-star ML repo by measuring it

What laya is, and why I was measuring it

The number that didn't fit

Making it hard to explain away

Someone else went deeper

Testing someone else's hypothesis, and my own

The maintainer's call

Handing over a tool

Before opening the PR

On using Claude Code

What I'd tell myself next time

Where it stands

131 is open, and the retrain is pending. When it lands, presentation_checks.py gives the first answer, and I'll run the labelled A–F and A/C/D before/after on top of it.

Claude Opus 5.5 vs GPT-6 Astra vs GPT-6 Sol: the effort knob matters more than the model

TL;DR

1. The price sheet (and its fine print)

2. Tokens per task is the hidden multiplier

3. Buy intelligence by the dollar

4. Benchmarks have referees

5. Where each model actually wins

6. Latency

7. Integration gotchas

8. A routing recipe you can steal

Caveats

Sources

I built a Jev-style decision model for Japanese in two days. Claude Code A/B-tested my architecture and killed it.

TL;DR

What a System One model is

The benchmark

Results for laya-multilingual

So I built one

The part I actually want to write about

Limits

Links

🛠️ I Built a One-Click ComfyUI Setup for RTX 5090 on Windows — No WSL2, No Docker

Why RTX 50-series Breaks Everything

The 5 Rules I Discovered

What I Built

What I Verified

Why Not Just Use WSL2 or Docker?

Wrapping Up

hiroki-abe-58 / ComfyUI-Win-Blackwell

ComfyUI for GeForce RTX 50-Series (Blackwell)

What Makes This Special

Technical Highlights

Why This Is Unique

How I Run 6 AI Services Simultaneously on RTX 5090 + WSL2 + Docker (And You Can Too)

The Problem Nobody Solved

The Root Cause

The Solution: gpu-run

The Dockerfile Template

Docker Compose Architecture

Design Decisions That Saved My Sanity

1. Docker Profiles for Resource Isolation

2. Shared Model Directory

3. Port Allocation Strategy

4. The torchaudio Trap

Lessons Learned (The Hard Way)

Current Stack (February 2026)

Is This Still Unique?

I Built a Voice Cloning GUI That Supports 10 Languages — Here's What I Learned Wrestling with CUDA on Windows published

What I Built

hiroki-abe-58 / Qwen3-TTS-JP

Japanese GUI + Whisper auto-transcription for Qwen3-TTS. RTX 5090 tested.

Qwen3-TTS-JP

Custom Voice -- Speech synthesis with preset speakers

Voice Design -- Describe voice characteristics to synthesize

Voice Clone -- Clone voice from reference audio

Settings -- GPU / VRAM / Model information

Related Projects

The Architecture in 30 Seconds

Getting It Running (It's Actually Easy Now)

What You Can Actually Build With This

Voice Cloning in 5 Lines

Design a Voice From Scratch

Cross-Lingual Cloning

Practical Use Cases I've Seen

GPU Compatibility

Things I Learned the Hard Way

131 is open, and the retrain is pending. When it lands, `presentation_checks.py` gives the first answer, and I'll run the labelled A–F and A/C/D before/after on top of it.

The Solution: `gpu-run`