1 Which workflow is a better fit for your project?
Getting a MetaHuman to speak involves more than matching lip movements to speech. Developers also need natural-looking facial animation and a practical way to connect their characters. In this test, we used the same character and audio in UE 5.6.1 to compare Mugen Morph's cloud workflow with NVIDIA Audio2Face-3D's local LocalA2F-Claire workflow.
If you want a more convenient character setup process and more active eyebrow, eye, and mouth-corner movements during speech, Mugen Morph is worth trying first. In our hands-on setup, the plugin's demo-scene and character-setup menu options made integration more straightforward. Raised eyebrows and lifted mouth corners in the default output also gave the character a stronger sense of engagement. With this cloud workflow, network conditions and request wait times are part of the experience.
NVIDIA Audio2Face-3D's local Claire workflow is an option for developers who need local inference and are willing to spend more time on configuration. It had a shorter wait before playback in the single run recorded here, but model deployment, animation blueprints, and the audio invocation flow required more setup. Upper-face movement was also more restrained in this test's default output.
New development preview with closer coordination between lip sync and expression
Alongside the plugin comparison, we are sharing a new round of facial-animation development results. The Chinese and English clips featuring two characters show continuous mouth movements together with smiles, tightened brows, and eyebrow raises. The difference between the happy and angry examples is particularly visible, making the characters' emotional expression more distinct while speaking.
These work-in-progress results show the kind of performance we aim to bring into Mugen Morph.
This article covers integration in UE and Unity, adaptation for non-MetaHuman characters, and both controlled comparisons and development demonstrations showing Mugen Morph's current output and future direction.
2 Test scope and conditions
Mugen Morph provides an audio-driven facial animation API and engine integration tools. In the UE workflow tested here, the plugin submits audio and applies the returned animation data to the character. On the NVIDIA side, we used the ACE plugin and the local Claire model. The configuration is shown below.
Both workflows used the same character and matching audio, with consistent camera, lighting, and parameter-selection practices. The lip-sync comparison was aligned to the start of speech; wait time was measured from the processing request.
3 Chinese and English lip-sync comparison in UE
Using the same character and matching Chinese and English audio for each comparison, we looked at the lip movements and facial performance produced by the two workflows. When watching, pay attention to whether the mouth follows the sounds, closes naturally during pauses, and coordinates with movement around the eyes, eyebrows, and mouth corners.
In these Chinese and English tests, both workflows generated lip opening, narrowing, and closure. Mugen Morph accompanied some lines with more upward movement at the mouth corners. NVIDIA Audio2Face-3D's Claire model produced a wider mouth opening at some points, with relatively less movement at the mouth corners.
The videos retain the full audio and synchronized comparison footage, so you can judge which lip movements and expression style better suit your character by listening to the actual speech.
4 Facial expression and the sense of engagement
In digital presentations and character dialogue, changes around the eyes, eyebrows, and mouth corners directly affect how engaged a character appears. In this comparison using longer Chinese and English audio, Mugen Morph showed noticeable eyebrow raises, blinks, and lifted mouth corners in addition to mouth opening and closure, giving the face a wider range of movement.
At approximately 00:11 in the Chinese video, the Mugen Morph character raises its eyebrows and opens its eyes wider. At around 00:25-00:26, the mouth corners lift more noticeably. In the English video at about 00:27, the character shows a clear smile; at around 00:31, the forehead and eye-area changes accompanying an eyebrow raise are also easy to see. By comparison, Claire, shown on the right, has less movement around the eyes and eyebrows at these points, with more of the motion concentrated around the mouth.
These details make the Mugen Morph character look more animated during continuous speech and can give it a warmer sense of engagement. For digital presenters, virtual hosts, or character-dialogue projects that call for more expressive faces, this is an advantage worth examining in these test results. Developers can judge whether the amount of expression suits their character's personality and dialogue style.
5 UE and Unity integration tutorials
In our integration work, Mugen Morph's convenience mainly came from two things: UE menu options for creating a test scene and configuring the selected character, and an importable Unity plugin with sample scenes. Developers can first get audio-driven animation working, then connect their own characters and projects. For teams using both UE and Unity, these integration paths make it easier to get started and reuse the workflow.
5.1 Mugen Morph integration in UE
- Prepare the plugin and project. This tutorial uses the UE 5.6 source plugin. Install the Visual Studio 2022 Game development with C++ workload, create a C++ project, and place the plugin in the project's Plugins/Mugen3D directory. Enable the plugin and the required dependencies, including MetaHuman and RigLogic where applicable. The initial source installation still requires a working build environment.
- Enter your API credentials. Open Edit → Project Settings → Mugen3D → MugenMorph API, then enter your Access Key and Secret Key. You can also open this page through Mugen3D → API Settings. Obtain your credentials from the developer page: https://mugen3d.ai/api/.
- Create a scene and configure the character. Run Mugen3D → Create MugenMorph Demo Level, then import your own MetaHuman or character with ARKit-compatible facial blendshapes. Select the character and run Mugen3D → Setup MugenMorph on Selected Actor.
- Select audio and play. Click Play and use Browse or the J key to select a WAV file. You can also record audio with Start Record / Stop Record. After the API returns, check synchronized playback of the audio and default facial animation.
This workflow groups common character-setup actions into menus, reducing the manual work of adding components and wiring the basic invocation logic when getting the sample running for the first time. Developers can then adapt it to their own project workflows.
5.2 Running the Mugen Morph sample in Unity
- Import the plugin. Create a Unity project that matches the sample, drag the downloaded MugenMorph.0.0.8.unitypackage into the editor, and click Import. This tutorial uses HDRP; other render pipelines require the corresponding sample and material configuration.
- Save the API settings. Open Tools → Mugen3D → API Settings, enter your Access Key and Secret Key, and click Save Settings.
- Open the sample scene. Open the MugenMorph scene under Assets → Mugen3D → Samples → Scenes, then click Play.
- Record or select audio. Use Start Record to record microphone audio, or Select File to choose an audio file. After processing finishes, check synchronized playback.
The sample provides a complete path from audio input and facial blendshape animation to synchronized playback. For non-MetaHuman characters that already have ARKit-compatible facial blendshapes, developers can also start from the sample and connect their own characters after completing the character and blendshape mapping. The reusable adaptation approach and engine entry points provide flexibility, while each character must still meet the relevant animation requirements.
Version note: These steps and screenshots correspond to the tutorial versions shown here and use the Mugen3D / MugenMorph names. For other versions, follow the plugin documentation for the applicable menus and directories.
5.3 NVIDIA integration in UE and the additional work for Unity
This NVIDIA tutorial uses the ACE plugin and a local model. In UE, installation and character import are followed by adding ACEAudioCurveSource, setting up the facial animation blueprint, and connecting audio invocation and trigger logic.
- Connect the animation blueprint. Add Apply ACE Face Animations to the facial animation blueprint before the ARKit mapping node, then connect the output pose.
- Configure audio invocation. Use Animate Character from Wav File Async, configure the WAV path and the appropriate A2F Provider, and set up an input event. The original tutorial uses the T key to trigger playback. Exposing parameters and switching models are optional extensions.
The NVIDIA official-plugin workflow used here targets UE. There is no equivalent ready-made official Unity plugin workflow that we could carry over directly for this tutorial. Using it in Unity would require additional integration for service or inference calls, facial-data mapping, and playback logic. We therefore did not produce a NVIDIA Unity integration tutorial for this comparison.
For projects involving both UE and Unity, or teams that want to validate an existing character using a ready-made sample, Mugen Morph's plugins and samples for both engines make it easier to get started. In this UE tutorial, menu-based character setup also reduces the steps needed to build the basic invocation chain manually. NVIDIA's workflow is better suited to developers willing to configure blueprints themselves who need local inference and more detailed parameter control.
Tutorials and documentation
- Mugen Morph API and developer access: https://mugen3d.ai/api/
- Mugen Morph Unity plugin and instructions: https://github.com/sumeurai/MugenMorph-For-Unity
- NVIDIA ACE UE plugin documentation for the referenced version: https://docs.nvidia.com/ace/ace-unreal-plugin/2.5/ace-unreal-plugin-audio2face.html
6 Which integration steps require less work?
In our integration experience, Mugen Morph's convenience came mainly from common UE character settings being grouped in the plugin menu, together with an existing plugin and sample scenes for Unity. Developers can complete basic validation first, then integrate audio-driven animation into their own projects.
These differences make Mugen Morph a practical option for developers who want to get a sample working first and then integrate it into a project step by step. In UE, menu options reduce manual setup of the basic character configuration and invocation logic. In Unity, the sample supplies a complete workflow that developers can inspect and reuse.
For teams using both UE and Unity, having an existing integration path in each engine is a direct practical advantage of Mugen Morph. Characters that already have ARKit-compatible facial blendshapes can also be adapted from the sample, allowing teams to reuse existing assets.
7 Wait time before playback and usage costs
In this round, we used the same Audio2Face.wav file to compare the wait before playback in Mugen Morph's cloud workflow and NVIDIA's local LocalA2F-Claire workflow. Timing starts when the processing-request log appears in the screen recording and ends when speech first becomes audible. It excludes the earlier time spent browsing folders and selecting audio.
In this single test, the local Claire workflow had a shorter wait, by approximately 25.9 seconds. The times above are estimates from the original recordings, and each timestamp refers to its respective source video.
This measures the overall time from the processing request to audible speech. It includes workflow processing, playback preparation, and any silence at the beginning of the input audio. It is not the model's pure computation time or the latency to the first frame of facial animation.
Both recordings completed playback of the test audio, with no observed application exit or playback stopping partway through.
Mugen Morph pricing by audio duration
As recorded on October 13, 2026, Mugen Morph's developer pricing had no subscription or monthly fee and used tiered billing based on the amount of audio processed during the month. Usage tiers reset on the first day of each month, and usage beyond a tier's threshold moves into the next rate. Check the live developer dashboard and account terms before publication or use.
| Monthly audio usage | Price |
|---|---|
| 1–6,000 seconds | $0.002 per second |
| 6,001–60,000 seconds | $0.0016 per second |
| 60,001–600,000 seconds | $0.0012 per second |
| Above 600,000 seconds | $0.001 per second |
At the first-tier rate, a 10-second clip costs approximately $0.02, and processing 3,600 seconds in a month costs approximately $7.20. With marginal tiered billing, 10,000 seconds would cost approximately $18.40: $12 for the first 6,000 seconds and $6.40 for the next 4,000 seconds. Budgeting should also account for retries, test clips, and repeated generations.
Hardware and licensing costs for NVIDIA's local workflow
NVIDIA's local workflow does not have the same per-second cloud API pricing as Mugen Morph. Official materials identify the Audio2Face-3D SDK's C++ and Python source code as MIT-licensed, while the model weights use the NVIDIA Open Model License. For product distribution, check the license files for the specific plugins, models, and dependencies you download.
For reference, the LocalA2F-James provider discussed in the hardware-cost information corresponds to the James v3.0 diffusion local provider. Official documentation lists a VRAM requirement of approximately 4.4 GiB or more, a supported NVIDIA Ampere, Ada, or Blackwell GPU, and the appropriate driver. Projects must also reserve VRAM for UE rendering, so being able to load the model does not guarantee that the target scene will run smoothly on the same device. This James hardware reference is separate from the Claire workflow used in the direct comparison above.
| Cost factor | Mugen Morph cloud workflow | NVIDIA LocalA2F-James hardware reference |
|---|---|---|
| Direct invocation costs | Charged by processed audio duration | No equivalent per-second API fee for the local path |
| Client GPU | No specific NVIDIA GPU required for facial inference | Supported NVIDIA GPU required; James v3 needs approximately 4.4 GiB or more of VRAM |
| Network | An available connection is required at runtime | Local inference does not depend on a remote A2F service |
| Engineering time | Mainly plugin compilation, credentials, character mapping, and network-error handling | Mainly plugins and models, AnimBP, blueprints, hardware compatibility, and performance tuning |
| Operations | Provider maintains the server; client manages quotas and retries | Team manages target devices, drivers, model packages, and version compatibility |
| When the cost model is attractive | Predictable usage and a desire to reduce local deployment work | Hardware already meets requirements and local usage is sufficient to justify engineering effort |
8 Which needs make Mugen Morph a good fit?
In this UE 5.6.1 test using the same MetaHuman character and default parameters, the two workflows had different strengths.
| Dimension | Findings from this test |
|---|---|
| Installation and integration | Mugen Morph was more convenient in our hands-on setup; the local Claire workflow required more configuration |
| English lip sync | Both produced lip movements, with differences in opening amplitude and closure timing; no overall accuracy ranking was established |
| Facial expression | Mugen Morph had more active eyebrows, eyes, and mouth corners; Claire's upper face was more restrained |
| Wait before playback | In the latest single recorded run, approximately 28.7 seconds for Mugen Morph and 2.8 seconds for Claire |
| Playback behavior | Both completed playback of the test audio in this run |
If your priority is getting a character connected in UE or Unity more quickly and you accept cloud calls, Mugen Morph is worth trying first. Its default output in this test also showed more noticeable movement around the eyes, eyebrows, and mouth corners.
If you prioritize local inference and are willing to invest in installation and configuration, LocalA2F-Claire is worth considering. In the latest test, it had a shorter wait from the processing request to speech playback.
There is no single winner across every dimension in this comparison. Integration convenience, lip-sync accuracy, expression style, and playback speed should be evaluated separately against your project's needs.
The direct comparison covers only one MetaHuman character, a limited set of audio samples, and default parameters. It does not represent all characters, speaking rates, or delivery styles. The NVIDIA workflow tested was local LocalA2F-Claire; the findings cannot be directly generalized to James, Mark, or cloud deployments. We did not score accuracy phoneme by phoneme, evaluate emotion recognition, quantify jitter, or test long-term stability.
9 Character adaptation and reuse of existing assets
For projects with existing character assets, a practical question is whether those characters can be reused for audio-driven animation and what adaptation is required.
The tutorials above show Mugen Morph integration paths in UE and Unity. Beyond MetaHuman, non-MetaHuman mesh characters with the required facial blendshape data can also use the relevant samples as a starting point, allowing existing characters to generate lip movements and expressions from audio.
This provides more options for reusing character assets. Actual integration still requires adaptation based on the character's facial blendshapes, parameter mapping, and plugin requirements. Developers can validate their own character with a short audio clip before integrating the full project workflow.
10 Development progress and future directions
Preview of the next Mugen Morph Mesh upgrade
Building on the current version, we are developing the next generation of our own Mugen Morph Mesh facial-animation system. The examples below show the new version's facial animation, with a focus on making lip movements, the eye and brow area, and overall expression work together more naturally and continuously during speech.
This demonstration uses two different characters and Chinese and English audio grouped into five emotions: Happy, Angry, Neutral, Fear, and Sad. Alongside the mouth movements, look at eyebrow raises and tightening, eye opening and closure, changes at the mouth corners, and the face as a whole to see how different emotions appear while the characters speak.
The overview image collects representative frames from the five emotion groups. The full videos show how the expressions change over the course of the speech. With this upgrade, we aim to help characters both follow the audio and convey more of the emotion in the delivery.
These are demonstrations of an unreleased development version. The earlier comparison uses the current plugin version.
Expanding to more types of characters
While continuing to improve facial animation for mesh characters, we also aim to gradually extend these capabilities to more character formats, including research into driving 3DGS characters.
Our goal is to let more kinds of character assets produce natural lip movements and facial expressions from speech, while fitting more easily into developers' existing creative workflows. We will continue sharing demonstrations and integration approaches as development progresses.
11 Frequently asked questions and developer access
Which is a better fit for UE developers, Mugen Morph or NVIDIA Audio2Face-3D?
If you want a quicker MetaHuman setup and accept cloud calls, try Mugen Morph first. If you want local inference and are willing to take on more configuration, consider NVIDIA LocalA2F-Claire. This article compares those two specific workflows.
Which has more accurate English lip sync?
We have not established an overall accuracy ranking in this round. Both produce opening and closing movements, but some movement amplitudes and timings differ. A wider mouth opening or richer expression does not automatically mean more accurate matching to the sounds being spoken.
Does more active facial expression mean better emotion recognition?
That cannot be concluded directly. The more visible eyebrow raises and lifted mouth corners in the UE videos are observable differences in expression. Emotion recognition was not separately evaluated in this test, and the five groups in the development demonstration do not constitute an emotion-recognition score.
Which workflow has a shorter wait before playback?
In the latest single test using Audio2Face.wav, Mugen Morph took approximately 28.7 seconds, and LocalA2F-Claire approximately 2.8 seconds. Timing starts when the processing-request log appears and ends when speech first becomes audible. These are wait times for the workflows in this test, not pure model computation times or a general performance ranking.
Is it fair to compare a cloud workflow with a local workflow?
It is possible to compare the integration steps, output, and wait times that developers experience, provided the deployment methods are explicit. Mugen Morph used the cloud here, while Claire ran locally. The results describe the overall workflows, and not every difference can be attributed to the models themselves.
Does easier integration mean that no build environment is needed?
No. The GitHub source-installation path used here involves Visual Studio and a C++ project. The convenience mainly concerns configuration, character setup, and the demo workflow after the plugin is enabled. For other installation methods, follow the instructions for that version.
Can the UE findings be applied directly to Unity?
No. The output comparisons and wait-time measurements in this round come from UE 5.6.1. Unity plugin integration, character adaptation, and runtime behavior need separate validation in the corresponding environment.
Can the development-preview capabilities already be used through the plugins?
The next-generation Mugen Morph Mesh version shown here has not yet been released. Use the current plugin version to judge what is available today. Exploration of driving 3DGS characters is a separate development direction; its integration should not be treated as already available through the current UE and Unity mesh-character workflows.
If you are choosing an audio-driven animation workflow, start with the existing plugin samples and use your own character and a typical audio clip to evaluate lip sync, facial style, and the setup experience.
Developer access: https://mugen3d.ai/api/
Top comments (0)