I have a habit of starting projects because I want one very specific thing.
Then somewhere along the way I apparently decide, "Well, if I'm alread...
For further actions, you may consider blocking this person and/or reporting abuse
The lineage tracking is the part most people skip and it's the part that actually matters. I built something similar for image generation last year and the "which params produced the one I liked" problem ate more time than the generation itself — ended up with 600+ files in a flat directory before I gave up and wired up a database. 40 GB of models is brutal but honestly sounds about right for this scope.
Hello Salika,
I hope you're doing well.
I have a good business idea that I'd love to discuss with you in more detail.
To give you some background, a friend of mine started this business with a U.S.-based partner three years ago. Since then, he's been paying his partner between $8,000 and $10,000 per month, and the business has been working well.
If you're interested in learning more, I'd be happy to share the details.
Whatsapp: +81 70-9427-3751
Telegram: @ono0319
Best regards,
Ono
A repaint can improve the selected section and still fail at its edit boundaries because the replacement may not share the original phase or ambience. Did you test those joins separately from the section's musical quality?
That’s a really good point. I tested the repaint results mostly for prompt responsiveness and whether the replacement itself behaved the way I expected, but I didn’t separately measure the edit boundaries for phase/ambience continuity.
In practice I’ve mostly been judging the joins by ear so far, especially on shorter repaint regions. That’s definitely something worth testing more deliberately though, because a musically good replacement can still sound wrong if the seam gives it away.
okay yeah this is so cool😭
the whole lineage/take history thing is such a smart choice because AI workflows get messy SO fast. being able to look back and actually understand “wait, which prompt / stem / take got me here?” is huge.
also massive respect for normalizing all those completely different model interfaces into one workflow because that is exactly the kind of engineering problem nobody thinks about until they try to actually build something usable lol
and the “sometimes software can just do the normal thing” philosophy is honestly my favorite part. trimming, fading, exporting, etc. absolutely do not need AI sprinkled on top just because AI is involved somewhere else. it makes the whole project feel way more intentional instead of AI-for-the-sake-of-AI.
I'm looking forward to working with local models, though I have no clue where to start lol!
seriously cool project 💚
Ahh thank you 💖 and YES, the lineage/history thing became way more important than I expected once I started actually using it. AI audio gets messy fast when you’ve got generations, repaints, stems, converted vocals, mixes, etc. and suddenly you’re like “wait... what the hell did this come from?” 😂
And the different model interfaces were definitely one of those “this seems simple until you actually build it” problems. They all want basically the same musical ideas expressed in completely different ways, so making that feel like one coherent workflow was a huge part of the project.
Also I’m 100% with you on the normal-software part. I really didn’t want to shove AI into every single feature just because the app uses AI elsewhere. Sometimes trim should just trim lol.
And honestly local models are way less scary once you start playing with them. I have a feeling you’d have a ridiculous amount of fun with them 😂💖
The "tiny experiment that got out of hand" pattern is one I recognize from my own projects. I started with a simple eval script and ended up with a full pipeline. The local-first approach is interesting for evals too: running offline means you can test without worrying about API rate limits or data privacy. What was the hardest part of making the local models good enough for real use?
Honestly, the hardest part wasn’t really making the models “good enough” so much as making them predictable enough to build around.
A lot of the work was figuring out what the models and runtime actually respond to versus what the docs imply. I hit cases where parameters were accepted but silently ignored, routes behaved differently than their names suggested, and memory usage was way higher than the model file size made it look. ACE-Step, for example, really needs its memory-saving mode on a 16 GB card, and some other families need their components loaded in a very particular way just to fit.
The other big part was normalizing all the little incompatibilities around them, like sample rates, prompt formats, output types, and task-specific request shapes, so the user doesn’t have to think about any of that. Miso ends up translating one workflow into whatever each model family actually expects.
So I’d say the hardest part was less “make the models better” and more “make the weirdness around the models disappear enough that the app feels reliable.”
The "documentation says X but the model actually responds to Y" line hits home - ran into the same thing wiring local model backends into CogniRunner, where a parameter was silently ignored unless it was nested exactly where the model's own reference implementation expected it, not where the API schema claimed it belonged. The Docker split keeping playback and editing alive while the inference container is down is the right call too - a lot of local-AI tools tie the UI to the same process serving the model, and the whole thing locks up the moment a generation job backs up.
Yes, exactly. The silent-parameter thing is especially nasty because nothing necessarily fails. You get a valid result back, it just completely ignored the setting you thought you changed, which can waste a ridiculous amount of time.
And that separation between Miso and the inference backend ended up being one of the better architectural decisions in the project. I really wanted the library, playback, editing, exports, etc. to still feel like normal software even if the model server is busy or completely down.
Local model tooling is full of these tiny integration traps that only show up once you stop doing single happy-path calls and try to build an actual usable app around them 😅
The audio.cpp-on-a-separate-machine split is the detail worth stealing - browser talks to Miso, Miso talks to inference, so the app half doesn't care whether the GPU box is even up. I've been doing something close to that splitting compute across two Macs on a LAN, and decoupling the transport like that is what makes the rest (editing, export, playback) keep working when the model server doesn't.
On the repaint quirk - sounds like repaint starts denoising from something closer to the original waveform than a fresh noise draw would, so there's less room left for the prompt to move it. Did ACE-Step's code confirm that, or is it still inferred from the opposite-prompt test?
Yeah, the backend split has turned out to be one of those decisions that keeps paying for itself. Once playback, editing, export, project state, etc. don’t depend on the inference process being healthy, the whole app feels a lot less fragile.
On repaint, that’s still an inference from behavior/testing rather than something I confirmed directly in ACE-Step’s code. The opposite-prompt test was what really convinced me the surrounding audio is doing most of the steering there, but I haven’t traced the repaint implementation deeply enough to say exactly how the starting latent/noise state is constructed.
Your explanation would fit the behavior really well though, so now I kind of want to go dig through that path and see if that’s actually what it’s doing.
Sharded the model across two GPUs after wiping 12GB VRAM to avoid OOM; the audio tokenizer’s hallucination trap was the real nightmare. Any other local AI music hacks you swear by?