DEV Community

AI-KSK
AI-KSK

Posted on

MiniMax H3 RefMod in ComfyUI: Build a Reusable Character, Motion, Voice and Scene Library (6 Workflows)

#ai #comfyui #machinelearning #tutorial

MiniMax H3 RefMod in ComfyUI: Build a Reusable Character, Motion, Voice and Scene Library

What if a recurring actor, a fight choreography reference, a character voice and a familiar set could be prepared once, then selected again for another shot? That is the production idea behind MiniMax H3 RefMod. Instead of repeatedly feeding every original image, video and sound file into every generation graph, these workflows extract reusable, pre-encoded reference assets and load them when rendering a new clip.

This is not LoRA training, model finetuning or a newly trained MiniMax H3 checkpoint. RefMods store reference representations used to condition an existing H3 model. The output still depends on the prompt, sampler, source quality, available reference tokens and the model itself. Reusable does not mean lossless, perfectly identity-locked or guaranteed to reproduce complex movement.

This AI-KSK tutorial is one episode with six complementary ComfyUI JSON workflows, rather than six unrelated demos. Workflows 01–04 create references; workflows 05–06 generate video from saved references. The settings below were inspected from the distributed JSON files and cross-checked against the supplied episode transcript. This written guide does not claim a new runtime test, measured speedup, observed frame quality or independent viewing of the published video.

The six workflows at a glance

No. Workflow Input route in this JSON Result / role
01 Character RefMod Folder loader connected; manual image input available but unconnected Save character/Rin_Identity for recurring visual identity
02 Motion RefMod Video Helper Suite VHS_LoadVideo connected Save motion/Action_01 as a motion-oriented reference
03 Audio RefMod LoadAudio → H3 audio VAE Save voice/Rin_Voice_01 from an audio clip
04 Dojo scene RefMod Two connected LoadImage nodes Save scene/Dojo_OneLocation from two viewpoints
05 Generation A — Apply Four saved refs injected after native prompt conditioning H3 audio/video synthesis via MiniMaxH3RefModApply
06 Generation B — Text Encode Three saved refs plus character/war loaded into reference-aware text encoder H3 audio/video synthesis with an empty prompt that needs writing

The workflow files are supplied together in the AI-KSK collection repositories, including the six illustrated project covers. These covers are promotional illustrations, not generated video frames or benchmarks.

1. What you need before opening a graph

Start with a compatible ComfyUI installation that supports MiniMax H3, the H3 video/audio VAE interfaces and the workflow's custom nodes. Import the JSON using ComfyUI's workflow loader. If nodes appear missing, resolve the node packages and restart/reload ComfyUI; do not substitute similarly named nodes from unrelated projects without checking their inputs.

Primary RefMod nodes: ComfyUI-MiniMaxH3Mod by Luisacaotica supplies MiniMaxH3RefModExtract, MiniMaxH3RefModAudioExtract, MiniMaxH3RefModFolderLoader, MiniMaxH3RefModsLoader, MiniMaxH3RefModApply, MiniMaxH3RefModTextEncode and MiniMaxH3RefModInspect.

Other nodes visible in the JSON: core ComfyUI loaders and sampler nodes (VAELoader, CLIPLoader, UNETLoader, BasicScheduler, BasicGuider, RandomNoise, SamplerCustomAdvanced, VAEDecode, VAEDecodeAudio, etc.) and Video Helper Suite nodes VHS_LoadVideo and VHS_VideoCombine. The source project for the latter is ComfyUI-VideoHelperSuite. Install appropriate versions through their respective upstream projects/ComfyUI Manager. The actual deployed image may already have them installed, but this has not been independently verified here.

The generator graphs select the following exact checkpoint filenames, not an interchangeable generic “H3 model”:

JSON usage Selected file
Generation video model minimax_h3_ref2va_pruned_int8_convrot.safetensors
Qwen3-VL text encoder qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors
Video VAE, creators and renderers minimax_h3_video_vae_int8_convrot.safetensors
Audio VAE minimax_h3_audio_vae_fp32.safetensors
Model-only LoRA for both generators minimax_h3_ref2v_turbo_4step_v0.1_comfyui_bf16.safetensors (strength 1)

Important: the Comfy-Org H3 model listing documents the corresponding H3 diffusion model, Qwen encoder and minimax_h3_video_vae_fp16.safetensors, but not necessarily the exact ...video_vae_int8_convrot... VAE and turbo LoRA used in these JSON files. Do not assume all filenames are available from one repository or rename a different VAE to silence a missing-model warning. Check the actual image inventory or the appropriate upstream checkpoint source and its terms. No third-party weights, saved RefMod assets or input media are distributed in this workflow collection.

Official/author project references:

Review the upstream license and territorial/use restrictions before downloading or operating the weights. The workflow collection does not relicense those components.

2. Workflow 01 — Character RefMod: images into a reusable actor

Purpose. Prepare one identity-oriented asset from several images of the same character, retaining its appearance as a reference for later shots. This episode's example character is called Rin.

What is actually wired. MiniMaxH3RefModFolderLoader feeds refs_bundle on MiniMaxH3RefModExtract, which uses the H3 video VAE. The extractor output goes to MiniMaxH3RefModInspect and PreviewImage. The extractor also exposes direct image/video sockets, but neither is connected in this supplied JSON. Consequently the saved graph as provided is folder-driven; the one-image alternative described in the video requires manually connecting a compatible image loader or modifying the workflow.

Exact JSON settings: folder Rin_Identity; max_items=32, max_frames=240, max_edge=1024. The extractor sets name=Rin_Identity, mode=Full Reference, concept_type=identity, ref_resolution=768, pool_h=16, pool_w=16, latent_frames=16, motion_only=false, multiplier=1, max_tokens=0, save=true, extraction_preset=manual, subfolder=character, budget_policy=truncate. The VAE loader requests minimax_h3_video_vae_int8_convrot.safetensors. The descriptive text specifies Rin's curled dark-brown updo, pearl earrings, ivory satin blouse, brown skirt and accessories. These are configuration values, not a quality guarantee.

Procedure:

  1. Put your own rights-cleared character photographs into a folder under ComfyUI's input directory, then set the folder loader's folder field to that folder's actual name. The sample Rin_Identity media folder is not included in this package.
  2. Confirm the image count and frame/edge caps for your material. These are upper bounds, not a promise that all files will be consumed in every circumstance.
  3. Review the identity description, concept type and VAE selection. Replace the Rin description if using another character.
  4. Execute the graph; inspect the output and check for a newly saved RefMod under ComfyUI/models/refmods/character/ according to the installed node's saving implementation. A path written in a tutorial is not proof the file already exists on your machine.
  5. If later loading character/Rin_Identity, select the asset you actually created, or update the generator loader to your own saved name.

Limits: Encoding references does not learn a permanent actor identity like an independently trained LoRA. Different viewpoints and costume variations may help the reference describe a character, but can also introduce conflicts.

3. Workflow 02 — Motion RefMod: action footage as movement reference

Purpose. Separate the idea of a movement sequence from the scene in which it was filmed, allowing a new character to perform related actions in a different environment.

What is wired. A single VHS_LoadVideo provides image frames to the extractor's refs_video.ref_video_0, then to inspect/preview nodes. There is a second video input socket but no connected folder batch loader in this JSON. The episode discusses a folder route as an alternative, but it must be added/configured by the operator; do not present batch input as already connected.

Exact JSON settings: the video loader points at 203719-922186723_small.mp4; force_rate=0, original dimensions (custom_width=0, custom_height=0), frame_load_cap=121, skip_first_frames=0, select_every_nth=1. The extractor is named Action_01, with Full Reference, concept_type=pose_motion, ref_resolution=512, pool_h=16, pool_w=16, latent_frames=56, motion_only=false, multiplier=1, save=true, subfolder=motion, budget_policy=truncate. Its description emphasizes turns, steps, rotation and weight transfer, and explicitly says it is not skeleton-only extraction.

Procedure:

  1. Upload your own motion clip to ComfyUI and replace the JSON's missing example .mp4 selection. The original clip is not bundled.
  2. Review whether the sampled frames capture the complete action. The loader's 121-frame cap and the extractor's 56 latent-frame setting describe different stages; do not equate them with seconds or a measured motion length.
  3. Keep pose_motion for a motion reference. If unwanted clothing/body appearance transfers from source footage, try the optional motion_only control and compare results; the actual distributed value is false, not true.
  4. Run the graph, inspect the result and locate the saved motion/Action_01 RefMod.

Limits: This is reference guidance, not a retargeted skeleton or independent actor-specific rig. Contact choreography, multiple performers and highly occluded poses may remain ambiguous.

4. Workflow 03 — Audio RefMod: an audio asset without a video loader

Purpose. Prepare a standalone audio reference for later H3 generations. It can be helpful in a voice-oriented scene, but the saved reference does not promise an exact voice clone, timing synchronization or isolation of speech from background music.

What is wired. LoadAudio → MiniMaxH3RefModAudioExtract with an H3 audio VAELoader; inspect output → PreviewAudio. The selected audio filename is 女主.mp3, which is not included. The audio extraction node handles audio; the visual folder reader used in 01 is not a substitute for MP3/WAV loading.

Exact JSON settings: name=Rin_Voice_01, max_seconds=8, max_tokens=0, budget_policy=truncate, concept_type=voice, subfolder=voice, save=true. The VAE is minimax_h3_audio_vae_fp32.safetensors. The node's own description says that speaker identity transfer and synchronization are not guaranteed.

Procedure: Upload an appropriate, authorized and preferably clean voice recording, select it in LoadAudio, check the eight-second extraction limit, run the extractor and inspect the reference under models/refmods/voice/. Use audio whose background content you would not mind influencing a future generation. In the supplied narration, AI-KSK specifically notes that music left in an example voice reference contaminated a later attempt. Pre-cleaning speech is therefore an operational recommendation, not evidence that the JSON removes music automatically.

Preview warning: MiniMaxH3RefModInspect is configured for preview=stored and strength=0.5; the episode says the audible preview may be short. That does not change the upstream max_seconds=8 extraction setting. A preview excerpt cannot establish what a model internally retained.

5. Workflow 04 — Scene RefMod: two views of one dojo

Purpose. Retain spatial and stylistic cues for a recurring indoor location, so later shots may share the same wooden beams, lanterns, lattice windows and props.

What is wired. Two LoadImage nodes are connected to the refs_image inputs of MiniMaxH3RefModExtract. Unlike workflow 02, the two references are actually wired. Their serialized file selections are Gemini_Generated_Image_6f1h0k6f1h0k6f1h.jpg and Gemini_Generated_Image_j0kir9j0kir9j0ki.jpg. Neither JPEG is distributed.

Exact JSON settings: name=Dojo_OneLocation, mode=Full Reference, concept_type=background, ref_resolution=1024, pool_h=16, pool_w=16, latent_frames=16, motion_only=false, save=true, subfolder=scene. The description asks for one empty wood-built dojo, consistent light, a ceremonial drum and weapon racks.

Procedure: supply two actual, authorized images of the same location with useful, complementary views; replace both absent JPEG selections; keep the environmental description aligned with the new images; execute and inspect the resulting scene/Dojo_OneLocation asset. The promotional dojo cover is not a substitute for the two missing input images. Neither two-view conditioning nor the prompt guarantees geometry-accurate multi-angle consistency.

6. Workflow 05 — Generation A: Apply saved references after text conditioning

This graph is the first synthesis pipeline. It uses the native MiniMaxH3ReferenceToVideo node to create text conditioning and an initial latent, and then sends the conditioning through MiniMaxH3RefModApply with a bundle from MiniMaxH3RefModsLoader. This order is key: the text encoder does not separately re-read the original images that were used to create each saved RefMod. The loaded reference tokens are injected after the native conditioning stage.

Actual loader slots:

  1. character/Rin_Identity, strength 1, copies 1.
  2. motion/Action_01, strength 1, copies 1.
  3. scene/Dojo_OneLocation, strength 1, copies 1.
  4. voice/Rin_Voice_01, strength 1, copies 1.

Remaining slots select (none). Apply is configured with override=false, retention=1, curve_direction=constant, scramble_seed=-1 and max_total_tokens=0. The generator's native direct image/video/audio inputs are unconnected; the selected saved RefMods are the referenced media route in this graph.

Exact render settings: width=832, height=480, length=243 frames, ref_image_size=match. RandomNoise=9527 (fixed), BasicScheduler=beta / 4 steps / denoise 1, KSamplerSelect=euler; the LoraLoaderModelOnly strength is 1. VHS_VideoCombine requests 24 fps, H.264 MP4, yuv420p, CRF 19, with output prefix AIKSK_REF05_APPLY_. 243 frames / 24 fps corresponds to 10.125 seconds nominally; this is arithmetic from the graph, not a newly measured runtime or published clip duration.

The existing full prompt stages a solitary Rin in the dojo: guarded punches, a rotating roundhouse kick, a short finishing combination and a spoken Mandarin line, “看清楚了吗？再来一次。” It explicitly excludes extra people and background music, and specifies the intended motion-to-character transfer. Use that prompt as an instruction to the model, not as evidence that the final output met every detail.

Procedure: first generate the three visual and one audio RefMods using 01–04. Confirm the files exist under models/refmods/ and that the four loader dropdowns resolve; refresh node definitions if freshly saved entries do not appear. Confirm all checkpoint/LoRA selections are available, review rights and text prompt, then run 05. Examine the actual MP4 for costume stability, physics, unintended speech/music and motion legibility. The graph metadata contains a preview filename and duration from the originating author's serialized state; the corresponding MP4 is not in this package and its metadata is not an independent result verification.

Strength caution: Reference influences depend on both loader strengths and Apply settings. More copies or tokens can increase computation and memory cost; there is no universal strength configuration that always solves identity or choreography.

7. Workflow 06 — Generation B: reference-aware Text Encode

This graph makes a different architectural choice. MiniMaxH3RefModsLoader feeds MiniMaxH3RefModTextEncode alongside the Qwen3-VL H3 text encoder and the H3 video VAE. The text-encode node outputs conditioning with references already attached. Its conditioning goes straight to BasicGuider; there is no MiniMaxH3RefModApply in the active path. Do not add Apply on top of the same reference bundle or the same references may be injected twice.

The advantage of this design is that a prompt can address the encoder's reference map, using labels such as <Video 1> and <Picture 1> rather than relying only on a general prose identity description. A stack of multiple images saved as one video-kind RefMod can be mapped as one Video label; one single-image asset may become a Picture label. The exact mapping depends on the saved bundle, its order and active entries. Read the map from the node's reference_map output or the separate inspector; do not hardcode the labels solely from the original filenames.

Actual configured loader slots:

  1. character/Rin_Identity;
  2. motion/Action_01;
  3. scene/Dojo_OneLocation;
  4. character/war.

Each is configured with strength 1 and copies 1. The last item, character/war, has no matching creator JSON or packaged RefMod, so the operator must supply a compatible saved second-character reference or change that slot. There is no voice/Rin_Voice_01 slot selected in this original B generator. Even though an audio VAE is connected for output decoding, that is not the same as applying the separate voice asset.

Critical setup difference: MiniMaxH3RefModTextEncode has a blank prompt ("") in the distributed JSON, reference_fps=24 and max_total_tokens=0. The graph is not supplied with a complete two-person fight prompt. Before generation, write a prompt that uses the actual observed reference map, specifies who attacks/defends, camera blocking and shot timing. For example, after checking the map, describe Rin's position and the second fighter's position using their verified <Video n>/<Picture n> labels. This is a prompting method, not a claim that the original JSON already contains those instructions.

Other render settings: EmptyMiniMaxH3LatentAV=832×480×73 frames, fixed seed 9527, beta scheduler with 4 steps, euler sampler, 24 fps H.264 MP4 with prefix AIKSK_REF06_TEXT_. At 24 fps 73 frames is about 3.04 seconds nominally. This contrasts with the episode's spoken two-fighter example and with workflow 05's longer 243-frame setup; do not copy 05's duration into 06.

There are two RefMod loader nodes in JSON 06. Node 26 feeds the text encoder; node 34 feeds a separate inspector/reference-map display. The latter is for inspection and is not a second active injection route into the sampler. Confirm their selected files remain consistent if you edit them.

Procedure: produce the required assets, including your own second-character RefMod; fix any missing path; inspect the map; write a concise scene-specific prompt; review image/video type labels and shot assignments; confirm checkpoint files; then render and judge the output. Because it gives the text encoder a view of visual reference information, this path may require extra reference encoding work. Whether it is faster, slower or more accurate on a particular installation is not established by these JSON files.

8. Apply vs Text Encode: choosing the right path

Dimension Apply — workflow 05 Text Encode — workflow 06
Reference entry point Pre-encoded refs appended via MiniMaxH3RefModApply Saved refs consumed by MiniMaxH3RefModTextEncode
When text conditioning happens Native H3 conditioning first, Apply afterward Text encoding and reference-aware mapping together
Present active refs Rin + Action_01 + Dojo + Rin voice Rin + Action_01 + Dojo + war second character
Prompt Detailed and filled in Empty: author must provide a prompt
Initial latent Native MiniMaxH3ReferenceToVideo EmptyMiniMaxH3LatentAV
Saved video configuration 832×480, 243 frames @ 24fps 832×480, 73 frames @ 24fps
Best starting point Straightforward repeat shot with four prepared assets Explicit reference assignment after checking live reference map

For a single actor performing one documented action in a recurring dojo, begin with A because its reference selection and shot prompt are already filled in. For multiple actors and precise description of which supplied reference is which, explore B after supplying the missing asset and prompt. Neither route supplies a deterministic two-person skeletal binding system; you may still need simpler framing, separated interactions and iterative prompts to limit identity mixing.

9. Troubleshooting checklist

Missing RefMod nodes. Install the upstream ComfyUI-MiniMaxH3Mod node pack and restart/reload ComfyUI. A saved JSON only records the node classes; it does not bundle executable code.

Missing image, video or MP3. The filenames in workflows 02–04 are selections from one author's ComfyUI instance, not downloads. Replace them with your own uploaded files. Workflow 01's folder name must point to a directory under the local input location.

Dropdown can't find Rin_Identity or Action_01. Save the asset first; check the category directory, and reload node options. The same applies to character/war, which is not created by the six included graphs.

Missing checkpoint or LoRA. Use the exact file inventory and upstream documentation, rather than silently renaming weights or assuming an incompatible file is equivalent. In particular, confirm the selected int8-convrot video VAE and turbo LoRA independently.

Weak or mixed identity. Confirm correct RefMod order and loader strengths; reduce competing references or simplify the prompt. A saved reference is not a character lock.

Unwanted music or sound. Clean your voice source and review how the audio RefMod was created. An output audio decoder does not mean a desired voice RefMod was used; B's active loader lacks one.

A Text Encode run feels under-specified. It is: the provided Text Encode prompt is empty. Check its reference map, supply a shot, and avoid double-Applying refs already attached by the encoder.

Comparing speed or quality. Hold model files, GPU, frame count, seed, resolution, prompt and reference budget constant in a controlled benchmark. The provided A and B graphs do not hold frame count, input set or prompt constant, so they are not a fair numerical A/B performance test.

10. Deploy the AI-KSK image and follow the full episode

If you prefer to start from the creator's prepared environment rather than assembling the node stack manually, the primary deployment entry for this episode is:

H3｜RefMod免训练超级参考 — AI-KSK image on 优云智算 / CompShare

Open the image page, check the provider's currently available configurations and charges, deploy an instance, enter ComfyUI and import/select the relevant JSON workflow. Verify model availability and add your own input media and saved RefMods as described above. Disclosure: this image link contains an AI-KSK referral code; using it may support the creator. Availability, installed versions, billing and successful inference are not verified by this article.

Watch the complete six-workflow video on YouTube. For Chinese-language viewing, the full Bilibili episode covers the same production sequence. The AI-KSK image index is an additional directory, not a substitute for this episode's deployment link.

The practical takeaway is a library-and-shoot workflow: make the reusable character, movement, audio and scene references first; select the right saved asset files in the renderer; and then choose between direct Apply injection and reference-aware text encoding. The included files make that architecture inspectable and reproducible once each user's actual media and dependencies are in place, but do not by themselves establish an image-quality or performance claim.

Individual workflow downloads on Civitai

Collection repositories: Hugging Face · ModelScope.

Top comments (0)