<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Kim</title>
    <description>The latest articles on DEV Community by Kim (@kim7).</description>
    <link>https://dev.to/kim7</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4207670%2F537edc04-fd55-445a-ad4b-77e101b5cf48.png</url>
      <title>DEV Community: Kim</title>
      <link>https://dev.to/kim7</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/kim7"/>
    <language>en</language>
    <item>
      <title>Designing a Reliable Credit-Based AI Generation Workflow</title>
      <dc:creator>Kim</dc:creator>
      <pubDate>Mon, 16 Nov 2026 16:46:41 +0000</pubDate>
      <link>https://dev.to/kim7/designing-a-reliable-credit-based-ai-generation-workflow-2oib</link>
      <guid>https://dev.to/kim7/designing-a-reliable-credit-based-ai-generation-workflow-2oib</guid>
      <description>&lt;p&gt;When building an AI image or video application, the generation request is only one part of the overall system.&lt;/p&gt;

&lt;p&gt;Users also need clear feedback about processing status, generation costs, failed requests, and available results.&lt;/p&gt;

&lt;p&gt;For developers, these requirements introduce several design considerations.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Model Generation as an Asynchronous Job
&lt;/h2&gt;

&lt;p&gt;Image and video generation requests may require significant processing time.&lt;/p&gt;

&lt;p&gt;Instead of treating every generation as a single synchronous operation, it can be useful to represent the request as a job with explicit states.&lt;/p&gt;

&lt;p&gt;For example:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;pending&lt;/code&gt;: The request has been accepted.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;processing&lt;/code&gt;: The model is generating the output.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;completed&lt;/code&gt;: The output is available.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;failed&lt;/code&gt;: The generation did not complete successfully.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Each job should have a unique identifier so its status can be retrieved consistently.&lt;/p&gt;

&lt;p&gt;This also makes it easier to investigate failures and prevent users from accidentally submitting duplicate requests.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Separate Credit Calculation from Job Execution
&lt;/h2&gt;

&lt;p&gt;A credit based application should calculate the required credits before the request begins.&lt;/p&gt;

&lt;p&gt;Different models and settings may have different costs.&lt;/p&gt;

&lt;p&gt;For example, a generation price might depend on model selection, video duration, output resolution, or image count.&lt;/p&gt;

&lt;p&gt;The interface can display the required credits before the user confirms the operation.&lt;/p&gt;

&lt;p&gt;On the backend, credit reservation and final settlement should follow explicit rules.&lt;/p&gt;

&lt;p&gt;A reliable implementation should prevent duplicate charging when a client retries a request.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Handle Failed Generations Carefully
&lt;/h2&gt;

&lt;p&gt;Generation failures can happen for several reasons, including service interruptions and invalid inputs.&lt;/p&gt;

&lt;p&gt;The product should provide meaningful failure feedback.&lt;/p&gt;

&lt;p&gt;If credits are reserved before processing, the system needs a consistent refund or release mechanism when a request fails.&lt;/p&gt;

&lt;p&gt;An idempotent settlement operation helps protect against duplicate refunds or deductions during retries.&lt;/p&gt;

&lt;p&gt;It is also useful to record the relevant job state transitions for troubleshooting.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Design the Frontend Around Job States
&lt;/h2&gt;

&lt;p&gt;The interface should communicate the generation status without forcing the user to guess what is happening.&lt;/p&gt;

&lt;p&gt;While a request is processing, the application can display progress feedback and prevent accidental duplicate submissions.&lt;/p&gt;

&lt;p&gt;When generation completes, the user should be able to view, download, or reuse the result.&lt;/p&gt;

&lt;p&gt;When generation fails, the interface should explain the failure and provide an appropriate recovery action.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Track the Entire Workflow
&lt;/h2&gt;

&lt;p&gt;Developers can evaluate the experience by tracking more than successful model responses.&lt;/p&gt;

&lt;p&gt;Useful events include:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Generation form opened&lt;/li&gt;
&lt;li&gt;Generation submitted&lt;/li&gt;
&lt;li&gt;Generation completed&lt;/li&gt;
&lt;li&gt;Generation failed&lt;/li&gt;
&lt;li&gt;Result downloaded&lt;/li&gt;
&lt;li&gt;Credits reserved and settled&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These events can help distinguish interface problems from backend processing failures.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;A reliable AI generation application needs consistent job management, transparent costs, understandable error handling, and clear interfaces.&lt;/p&gt;

&lt;p&gt;These principles are especially relevant for AI video platforms, where processing takes longer and individual generation requests may be expensive.&lt;/p&gt;

&lt;p&gt;I'm exploring these product design questions while developing &lt;a href="https://molensai.com/?utm_source=devto&amp;amp;utm_medium=referral&amp;amp;utm_campaign=generation_workflow" rel="noopener noreferrer"&gt;Molens AI&lt;/a&gt;, an AI image and video creation platform.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: This article was prepared with AI writing assistance and requires author review. The technical recommendations describe a general implementation approach and do not claim to document Molens AI's internal architecture.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>productivity</category>
      <category>chatgpt</category>
    </item>
    <item>
      <title>Building a Browser-Based AI Vocal Remover: What I Learned About Audio Quality and User Experience</title>
      <dc:creator>Kim</dc:creator>
      <pubDate>Sun, 15 Nov 2026 09:33:59 +0000</pubDate>
      <link>https://dev.to/kim7/building-a-browser-based-ai-vocal-remover-what-i-learned-about-audio-quality-and-user-experience-26ck</link>
      <guid>https://dev.to/kim7/building-a-browser-based-ai-vocal-remover-what-i-learned-about-audio-quality-and-user-experience-26ck</guid>
      <description>&lt;p&gt;When I started working on VocalRemover.one, I wanted to make a common music editing task easier: separating vocals from a finished song without requiring users to install complicated audio software.&lt;/p&gt;

&lt;p&gt;The concept sounds straightforward. A user uploads a song, an AI model separates the voice from the instrumental, and the application provides downloadable audio files.&lt;/p&gt;

&lt;p&gt;However, building a useful product around audio separation raises several interesting questions about audio quality, user expectations, and interface design.&lt;/p&gt;

&lt;p&gt;Here are some of the lessons I have learned while developing the product.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Audio separation quality is highly dependent on the source
&lt;/h2&gt;

&lt;p&gt;A finished song is a combination of vocals, drums, bass, instruments, and effects.&lt;/p&gt;

&lt;p&gt;These sounds often overlap in frequency and time, which makes recovering individual sources challenging.&lt;/p&gt;

&lt;p&gt;AI source separation can estimate the different components of a recording, but the result is influenced by several factors:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Heavy vocal reverb and delay&lt;/li&gt;
&lt;li&gt;Layered backing vocals&lt;/li&gt;
&lt;li&gt;Dense instrumental arrangements&lt;/li&gt;
&lt;li&gt;Distortion and clipping&lt;/li&gt;
&lt;li&gt;Lossy audio compression&lt;/li&gt;
&lt;li&gt;Live recordings with crowd noise&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For example, a vocal might sound reasonably isolated during a quiet verse but still contain drum or synthesizer leakage during a loud chorus.&lt;/p&gt;

&lt;p&gt;This creates a product design challenge: how do you help users understand the quality of the result before they commit to using it?&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Previewing the output is an important part of the workflow
&lt;/h2&gt;

&lt;p&gt;One design decision I consider particularly important is allowing users to listen to the separated audio before unlocking the downloadable result.&lt;/p&gt;

&lt;p&gt;In &lt;a href="https://vocalremover.one/" rel="noopener noreferrer"&gt;VocalRemover.one&lt;/a&gt;, the workflow is organized around three steps:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Upload a supported audio file.&lt;/li&gt;
&lt;li&gt;Preview the separated vocal and instrumental outputs.&lt;/li&gt;
&lt;li&gt;Unlock the full result when the separation sounds suitable.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This gives users a chance to evaluate their own recording.&lt;/p&gt;

&lt;p&gt;A singer preparing a backing track might care about whether the lead vocal is still audible.&lt;/p&gt;

&lt;p&gt;A producer extracting an acapella might care about how much percussion remains behind the voice.&lt;/p&gt;

&lt;p&gt;Both users are working with the same separation technology, but they have different definitions of an acceptable result.&lt;/p&gt;

&lt;p&gt;The preview helps them make that decision.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. The output needs to match the user's task
&lt;/h2&gt;

&lt;p&gt;Another interesting lesson is that users often describe their goals using different language.&lt;/p&gt;

&lt;p&gt;Someone might search for a vocal remover, while another person searches for an instrumental maker or an acapella extractor.&lt;/p&gt;

&lt;p&gt;These queries can represent different goals:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Vocal removal:&lt;/strong&gt; Reduce the lead singer while preserving the music.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Instrumental creation:&lt;/strong&gt; Prepare a backing track for singing, rehearsals, or covers.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Acapella extraction:&lt;/strong&gt; Isolate a vocal for remixing, editing, or studying phrasing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Stem splitting:&lt;/strong&gt; Separate vocals, drums, bass, and other instruments for more detailed music work.&lt;/p&gt;

&lt;p&gt;This is why the product includes different workflows for different audio tasks.&lt;/p&gt;

&lt;p&gt;The separation technology can support multiple use cases, but the interface should help users choose the output that makes sense for their project.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. More output tracks introduce more expectations
&lt;/h2&gt;

&lt;p&gt;A basic vocal remover can be evaluated by listening to two outputs: vocals and instrumental.&lt;/p&gt;

&lt;p&gt;A four-stem splitter introduces additional outputs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Vocals&lt;/li&gt;
&lt;li&gt;Drums&lt;/li&gt;
&lt;li&gt;Bass&lt;/li&gt;
&lt;li&gt;Other instruments&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These categories are useful for remix preparation and music practice, but the separation is still an estimation of the original sources.&lt;/p&gt;

&lt;p&gt;The "other" stem, for example, may contain guitars, keyboards, synthesizers, and remaining musical material together.&lt;/p&gt;

&lt;p&gt;It is important to describe these outputs accurately so users know what they can expect.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Audio quality should be evaluated by listening
&lt;/h2&gt;

&lt;p&gt;A waveform can show amplitude, but it cannot tell the full story about whether a separated vocal sounds natural.&lt;/p&gt;

&lt;p&gt;When evaluating a result, I find it useful to consider questions such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Are the words and vocal phrasing still intelligible?&lt;/li&gt;
&lt;li&gt;Does the instrumental contain noticeable vocal remnants?&lt;/li&gt;
&lt;li&gt;Are drums and transients preserved?&lt;/li&gt;
&lt;li&gt;Does the result contain metallic or watery artifacts?&lt;/li&gt;
&lt;li&gt;Would the output actually work for the intended project?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These questions are especially important because the standard for acceptable quality changes between karaoke practice, remix drafts, and professional production.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. A useful AI product should explain its limitations
&lt;/h2&gt;

&lt;p&gt;AI audio tools can provide access to workflows that previously required specialist software and experience.&lt;/p&gt;

&lt;p&gt;At the same time, source separation cannot guarantee a perfectly clean result for every recording.&lt;/p&gt;

&lt;p&gt;Clear expectations, understandable output choices, and the ability to preview results are important parts of the experience.&lt;/p&gt;

&lt;p&gt;I am continuing to improve &lt;a href="https://vocalremover.one/" rel="noopener noreferrer"&gt;VocalRemover.one&lt;/a&gt; around these ideas.&lt;/p&gt;

&lt;p&gt;I would also be interested to hear how other developers approach AI media processing and how they communicate variable output quality to users.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When building an AI tool with unpredictable output quality, how much of the result should users be able to evaluate before paying?&lt;/strong&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>productivity</category>
      <category>chatgpt</category>
    </item>
    <item>
      <title>What I Learned Building Tiny Desk AI: The Challenges of Five-Person Video Generation</title>
      <dc:creator>Kim</dc:creator>
      <pubDate>Mon, 26 Oct 2026 04:48:34 +0000</pubDate>
      <link>https://dev.to/kim7/what-i-learned-building-tiny-desk-ai-the-challenges-of-five-person-video-generation-3bo6</link>
      <guid>https://dev.to/kim7/what-i-learned-building-tiny-desk-ai-the-challenges-of-five-person-video-generation-3bo6</guid>
      <description>&lt;p&gt;AI video generation has become increasingly capable of turning a single portrait into a realistic animated performance. However, generating a video with five different people, consistent appearances, and precise camera transitions introduces a completely different set of engineering challenges.&lt;/p&gt;

&lt;p&gt;I'm currently developing &lt;a href="https://tinydeskai.net/" rel="noopener noreferrer"&gt;Tiny Desk AI&lt;/a&gt;, a web application exploring personalized concert-style videos generated from user-uploaded photos.&lt;/p&gt;

&lt;p&gt;One of the most interesting experiments has been creating a five-person music performance in which every performer is assigned a different reference image.&lt;/p&gt;

&lt;p&gt;In this article, I'll share some of the technical problems I encountered and the approaches I've been exploring to improve the results.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. The Idea: Five Photos, Five Performers
&lt;/h2&gt;

&lt;p&gt;The concept is straightforward.&lt;/p&gt;

&lt;p&gt;A user uploads five portrait photos, and each photo is mapped to a specific performer in a music performance.&lt;/p&gt;

&lt;p&gt;The five roles include a featured lead singer, three supporting vocalists, and a saxophone player.&lt;/p&gt;

&lt;p&gt;The challenge is making sure that each person appears in the correct role throughout the entire video, including close-ups and camera transitions.&lt;/p&gt;

&lt;p&gt;For example, this is the intended mapping:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;characterMapping&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;leadSinger&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;picture1&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;vocal1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;picture2&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;vocal2&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;picture3&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;vocal3&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;picture4&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;saxPlayer&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;picture5&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The mapping looks simple in code, but a generative video model may not reliably maintain those assignments throughout the output.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. The First Problem: Identity Consistency
&lt;/h2&gt;

&lt;p&gt;One of the most noticeable issues occurs when multiple characters appear in the same scene.&lt;/p&gt;

&lt;p&gt;A model might correctly generate the first three performers while accidentally assigning the wrong identity to the saxophone player.&lt;/p&gt;

&lt;p&gt;Another common issue is identity drift across camera cuts. A performer can initially appear with the correct face and hairstyle, then gradually develop different features in a later shot.&lt;/p&gt;

&lt;p&gt;The prompts I tested explicitly required fixed character assignments and prohibited identity duplication.&lt;/p&gt;

&lt;p&gt;However, stronger instructions alone did not guarantee consistent results.&lt;/p&gt;

&lt;p&gt;This led me to investigate more structured character references and shot-specific generation workflows.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. The Second Problem: Preserving the Original Shot Timeline
&lt;/h2&gt;

&lt;p&gt;For one experiment, I used a reference performance lasting approximately 13.59 seconds.&lt;/p&gt;

&lt;p&gt;The video contained three principal segments:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Segment&lt;/th&gt;
&lt;th&gt;Duration&lt;/th&gt;
&lt;th&gt;Performers&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Opening group performance&lt;/td&gt;
&lt;td&gt;0 to 5.29s&lt;/td&gt;
&lt;td&gt;Three backing performers&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Featured singer close-up&lt;/td&gt;
&lt;td&gt;5.29 to 11.58s&lt;/td&gt;
&lt;td&gt;Lead singer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Saxophone performance&lt;/td&gt;
&lt;td&gt;11.58 to 13.59s&lt;/td&gt;
&lt;td&gt;Saxophone player and partially visible keyboard performer&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The initial generation configuration requested a 15-second result at 720p resolution.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"model"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"seedance-2.0-mini"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"duration"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"ratio"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"9:16"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"resolution"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"720p"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"generate_audio"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One generated result introduced an additional camera transition during the lead singer's segment and extended the final saxophone performance beyond the original reference.&lt;/p&gt;

&lt;p&gt;The difference between the reference duration and requested output duration was one possible contributing factor.&lt;/p&gt;

&lt;p&gt;I learned that preserving temporal structure requires more than describing the intended performance. It is also necessary to inspect whether the generation workflow can respect exact shot boundaries.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Building a Shot-Aware Generation Workflow
&lt;/h2&gt;

&lt;p&gt;An approach I'm exploring is representing the reference video as a sequence of predefined shots.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;referenceShots&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;start&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;end&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;5.292&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;performers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;vocal1&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;vocal2&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;vocal3&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;start&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;5.292&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;end&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;11.583&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;performers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;leadSinger&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;start&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;11.583&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;end&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;13.593&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;performers&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;saxPlayer&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;vocal1&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;];&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This representation provides a useful source of truth for validation.&lt;/p&gt;

&lt;p&gt;It helps answer questions such as which identities should appear in a particular shot, when the camera should switch, and whether a recurring performer is represented consistently.&lt;/p&gt;

&lt;p&gt;For example, the keyboard performer appears in the opening segment and becomes partially visible again in the final saxophone shot.&lt;/p&gt;

&lt;p&gt;That performer must retain the same assigned identity in both appearances.&lt;/p&gt;

&lt;p&gt;The metadata does not force a video model to follow these constraints automatically. It provides a foundation for evaluating results and potentially splitting generation into smaller, more controllable tasks.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Full Character Replacement Is More Complicated Than Face Replacement
&lt;/h2&gt;

&lt;p&gt;Another lesson involved the difference between facial identity and complete character appearance.&lt;/p&gt;

&lt;p&gt;In some generated results, the replacement person had recognizable facial features but retained the original performer's clothing or hairstyle.&lt;/p&gt;

&lt;p&gt;For a user expecting their uploaded character to appear naturally in the concert, this can create an inconsistent result.&lt;/p&gt;

&lt;p&gt;Full character replacement needs to consider facial features, hairstyle, visible body proportions, clothing, posture, and interaction with instruments.&lt;/p&gt;

&lt;p&gt;At the same time, the system should preserve the source performance's camera framing, choreography, and overall scene structure.&lt;/p&gt;

&lt;p&gt;There is a tradeoff here. Allowing the model to reconstruct more of the person gives it additional freedom, which can also increase unintended changes to the original performance.&lt;/p&gt;

&lt;p&gt;A promising direction is to separate identity and appearance references from motion and staging constraints, then evaluate each part independently.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. Evaluating Generated Videos
&lt;/h2&gt;

&lt;p&gt;I've found it useful to inspect results based on several dimensions instead of treating every completed generation as a success.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Identity accuracy:&lt;/strong&gt; Is each uploaded person assigned to the correct performer?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Character consistency:&lt;/strong&gt; Does the same person remain recognizable after a camera cut?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Temporal accuracy:&lt;/strong&gt; Are the original shot boundaries and performance duration preserved?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Visual consistency:&lt;/strong&gt; Do clothing, hairstyles, instruments, and character positions remain coherent?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Delivery reliability:&lt;/strong&gt; Can the generated video actually be saved, played, and downloaded?&lt;/p&gt;

&lt;p&gt;The last point is particularly important when turning an AI experiment into a web application.&lt;/p&gt;

&lt;p&gt;Generation APIs may return asynchronous tasks, fail after submission, or provide temporary result URLs. A production system therefore needs persistent task states, reliable file storage, and recovery mechanisms for unsuccessful generations.&lt;/p&gt;

&lt;h2&gt;
  
  
  7. What I Would Improve Next
&lt;/h2&gt;

&lt;p&gt;Based on these experiments, my next areas of investigation are explicit performer references, better control over original shot boundaries, and stronger validation of character consistency.&lt;/p&gt;

&lt;p&gt;For tasks requiring precise preservation of camera cuts and performance timing, processing shots individually before assembling the final video may offer better control than generating the entire clip in one request.&lt;/p&gt;

&lt;p&gt;However, that approach introduces additional complexity around motion continuity, editing boundaries, and audio synchronization.&lt;/p&gt;

&lt;p&gt;The right solution depends on the video model's editing capabilities and the required level of fidelity.&lt;/p&gt;

&lt;h2&gt;
  
  
  Final Thoughts
&lt;/h2&gt;

&lt;p&gt;Building multi-person AI video experiences has highlighted how difficult it can be to maintain consistent characters across a short sequence with multiple camera angles.&lt;/p&gt;

&lt;p&gt;Even when individual frames look convincing, identity drift, unexpected transitions, and small timing differences can affect the final result.&lt;/p&gt;

&lt;p&gt;My current focus with Tiny Desk AI is improving these aspects of the generation workflow and making the results more reliable.&lt;/p&gt;

&lt;p&gt;I'm curious whether other developers working on reference-to-video generation have encountered similar identity consistency issues, especially when five or more characters appear across different shots.&lt;/p&gt;

&lt;p&gt;What techniques have worked best for maintaining stable characters across camera cuts?&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Disclosure: I'm developing Tiny Desk AI, the project discussed in this article. AI assistance was used in drafting this post. The examples describe ongoing development experiments, and the suggested approaches are not presented as fully validated production solutions. Any use of people’s images, reference footage, or music should respect consent and applicable licensing requirements.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>productivity</category>
      <category>showdev</category>
    </item>
  </channel>
</rss>
