DEV Community: Mugen3D Dev

Same Line, Different Emotions: A Mugen Morph Character Performance Preview

Mugen3D Dev — Thu, 05 Nov 2026 08:27:32 +0000

One character image, one audio clip—and the same line expressed through different facial emotions.
The same words can sound joyful, sad, angry, or uneasy. The words stay the same, and so does the audio, yet the character’s eyes, brows, and mouth can convey a very different feeling.
I was fortunate to take part in an early test of Mugen Morph. For this review, I chose Phil Dunphy from Modern Family as the character. I wanted to see how his facial expressions would change when the same audio was paired with different emotion settings. I was pleased with the results, especially the way his eyes, brows, and mouth shifted to express different emotions.
Mugen Morph is developing a workflow for image- and audio-driven character performance. Upload a character image, provide audio, and choose an emotion style to bring a still character to life. Lip movements follow the speech, while facial expressions add another layer of feeling.
For longer passages, Mugen Morph can help maintain a natural, consistent performance from beginning to end, with expressions that carry through the delivery and keep the character engaged.
Mugen Morph also supports responsive real-time interaction. From an emotional monologue to an impromptu conversation, characters can perform—and respond in the moment.
In this preview, we’ll use the same line with different emotion settings to show how Mugen Morph brings different feelings to a character’s performance.
One line. Different emotions. One character, many ways to express.

The Technology Behind Speech-Driven Facial Animation
Speech-driven facial animation involves coordinating facial motion over time. Mouth movements need to align with speech, the performance needs to remain continuous, and facial details need to convey different expressions.
Research has explored these challenges from several directions. For example, FaceFormer investigates how audio context can be used to generate continuous 3D facial animation, while EmoTalk studies the relationship between speech content, emotion, and facial movement. Together, these approaches illustrate some of the technical challenges involved in speech-driven facial animation.
This preview focuses on the relationship between an emotion setting and the resulting facial expression, using the same audio input across the examples.
Demo Design: One Line, One Audio Clip, Seven Emotions
For this demo, we use the same character image and the same audio clip, then select seven emotion presets: Calm, Angry, Disgust, Fear, Happy, Sad, and Surprise.
The character image and audio remain unchanged; only the emotion setting changes. This makes it easy to compare how the character’s facial expression varies across the seven settings.

Use the same character image

Use the same audio clip for all seven outputs

Seven emotions preset

Seven generated expressions
Mugen Morph Workflow
Mugen Morph’s existing product materials include an audio-driven animation workflow powered by a cloud API, as well as an Unreal Engine plugin and a Unity sample project. This preview focuses on an image-and-audio-driven character expression workflow, highlighting how an emotion setting affects the resulting facial performance.
The demo workflow:

Upload a character image.
Provide or upload an audio clip.
Select an emotion.
Generate the character’s facial performance for presentation on a Mesh, in video, or as a 3D Gaussian Splatting (3DGS) character.

Example Use Cases
Image- and audio-driven character expressions can support a range of content workflows, including:

Digital presenters: Add a character presenter to a course slide, presentation, or explainer video.
Courses and educational content: Create character-led lessons, training materials, and knowledge-sharing content.
Character performance previews: Compare how one character performs with different emotion settings.
Creator content: Combine a character image and audio to create expressive character-led content.

The Character in This Demo
This demo features Phil Dunphy. Using a familiar character helps keep the focus on the facial performance: the character and audio stay consistent while the emotion setting changes.
This character is used solely for product testing. If you believe this use infringes your rights, please contact us for removal.
Coming Soon
Mugen Morph’s image- and audio-driven character expression feature is currently in development and is planned for release in mid-November. Join our community for launch updates and information about trying the feature.
Discord: https://discord.gg/a5drEtENt

Mugen Morph vs NVIDIA Audio2Face-3D Which audio-driven facial animation workflow is a better fit for developers?

Mugen3D Dev — Wed, 04 Nov 2026 11:17:12 +0000

1 Which workflow is a better fit for your project?

Getting a MetaHuman to speak involves more than matching lip movements to speech. Developers also need natural-looking facial animation and a practical way to connect their characters. In this test, we used the same character and audio in UE 5.6.1 to compare Mugen Morph's cloud workflow with NVIDIA Audio2Face-3D's local LocalA2F-Claire workflow.
If you want a more convenient character setup process and more active eyebrow, eye, and mouth-corner movements during speech, Mugen Morph is worth trying first. In our hands-on setup, the plugin's demo-scene and character-setup menu options made integration more straightforward. Raised eyebrows and lifted mouth corners in the default output also gave the character a stronger sense of engagement. With this cloud workflow, network conditions and request wait times are part of the experience.
NVIDIA Audio2Face-3D's local Claire workflow is an option for developers who need local inference and are willing to spend more time on configuration. It had a shorter wait before playback in the single run recorded here, but model deployment, animation blueprints, and the audio invocation flow required more setup. Upper-face movement was also more restrained in this test's default output.

New development preview with closer coordination between lip sync and expression

Alongside the plugin comparison, we are sharing a new round of facial-animation development results. The Chinese and English clips featuring two characters show continuous mouth movements together with smiles, tightened brows, and eyebrow raises. The difference between the happy and angry examples is particularly visible, making the characters' emotional expression more distinct while speaking.
These work-in-progress results show the kind of performance we aim to bring into Mugen Morph.

This article covers integration in UE and Unity, adaptation for non-MetaHuman characters, and both controlled comparisons and development demonstrations showing Mugen Morph's current output and future direction.

2 Test scope and conditions

Mugen Morph provides an audio-driven facial animation API and engine integration tools. In the UE workflow tested here, the plugin submits audio and applies the returned animation data to the character. On the NVIDIA side, we used the ACE plugin and the local Claire model. The configuration is shown below.

Both workflows used the same character and matching audio, with consistent camera, lighting, and parameter-selection practices. The lip-sync comparison was aligned to the start of speech; wait time was measured from the processing request.

3 Chinese and English lip-sync comparison in UE

Using the same character and matching Chinese and English audio for each comparison, we looked at the lip movements and facial performance produced by the two workflows. When watching, pay attention to whether the mouth follows the sounds, closes naturally during pauses, and coordinates with movement around the eyes, eyebrows, and mouth corners.
In these Chinese and English tests, both workflows generated lip opening, narrowing, and closure. Mugen Morph accompanied some lines with more upward movement at the mouth corners. NVIDIA Audio2Face-3D's Claire model produced a wider mouth opening at some points, with relatively less movement at the mouth corners.

The videos retain the full audio and synchronized comparison footage, so you can judge which lip movements and expression style better suit your character by listening to the actual speech.

4 Facial expression and the sense of engagement

In digital presentations and character dialogue, changes around the eyes, eyebrows, and mouth corners directly affect how engaged a character appears. In this comparison using longer Chinese and English audio, Mugen Morph showed noticeable eyebrow raises, blinks, and lifted mouth corners in addition to mouth opening and closure, giving the face a wider range of movement.
At approximately 00:11 in the Chinese video, the Mugen Morph character raises its eyebrows and opens its eyes wider. At around 00:25-00:26, the mouth corners lift more noticeably. In the English video at about 00:27, the character shows a clear smile; at around 00:31, the forehead and eye-area changes accompanying an eyebrow raise are also easy to see. By comparison, Claire, shown on the right, has less movement around the eyes and eyebrows at these points, with more of the motion concentrated around the mouth.
These details make the Mugen Morph character look more animated during continuous speech and can give it a warmer sense of engagement. For digital presenters, virtual hosts, or character-dialogue projects that call for more expressive faces, this is an advantage worth examining in these test results. Developers can judge whether the amount of expression suits their character's personality and dialogue style.

5 UE and Unity integration tutorials

In our integration work, Mugen Morph's convenience mainly came from two things: UE menu options for creating a test scene and configuring the selected character, and an importable Unity plugin with sample scenes. Developers can first get audio-driven animation working, then connect their own characters and projects. For teams using both UE and Unity, these integration paths make it easier to get started and reuse the workflow.

5.1 Mugen Morph integration in UE

Prepare the plugin and project. This tutorial uses the UE 5.6 source plugin. Install the Visual Studio 2022 Game development with C++ workload, create a C++ project, and place the plugin in the project's Plugins/Mugen3D directory. Enable the plugin and the required dependencies, including MetaHuman and RigLogic where applicable. The initial source installation still requires a working build environment.
Enter your API credentials. Open Edit → Project Settings → Mugen3D → MugenMorph API, then enter your Access Key and Secret Key. You can also open this page through Mugen3D → API Settings. Obtain your credentials from the developer page: https://mugen3d.ai/api/.
Create a scene and configure the character. Run Mugen3D → Create MugenMorph Demo Level, then import your own MetaHuman or character with ARKit-compatible facial blendshapes. Select the character and run Mugen3D → Setup MugenMorph on Selected Actor.
Select audio and play. Click Play and use Browse or the J key to select a WAV file. You can also record audio with Start Record / Stop Record. After the API returns, check synchronized playback of the audio and default facial animation.

This workflow groups common character-setup actions into menus, reducing the manual work of adding components and wiring the basic invocation logic when getting the sample running for the first time. Developers can then adapt it to their own project workflows.

5.2 Running the Mugen Morph sample in Unity

Import the plugin. Create a Unity project that matches the sample, drag the downloaded MugenMorph.0.0.8.unitypackage into the editor, and click Import. This tutorial uses HDRP; other render pipelines require the corresponding sample and material configuration.
Save the API settings. Open Tools → Mugen3D → API Settings, enter your Access Key and Secret Key, and click Save Settings.
Open the sample scene. Open the MugenMorph scene under Assets → Mugen3D → Samples → Scenes, then click Play.
Record or select audio. Use Start Record to record microphone audio, or Select File to choose an audio file. After processing finishes, check synchronized playback.

The sample provides a complete path from audio input and facial blendshape animation to synchronized playback. For non-MetaHuman characters that already have ARKit-compatible facial blendshapes, developers can also start from the sample and connect their own characters after completing the character and blendshape mapping. The reusable adaptation approach and engine entry points provide flexibility, while each character must still meet the relevant animation requirements.
Version note: These steps and screenshots correspond to the tutorial versions shown here and use the Mugen3D / MugenMorph names. For other versions, follow the plugin documentation for the applicable menus and directories.

5.3 NVIDIA integration in UE and the additional work for Unity

This NVIDIA tutorial uses the ACE plugin and a local model. In UE, installation and character import are followed by adding ACEAudioCurveSource, setting up the facial animation blueprint, and connecting audio invocation and trigger logic.

Connect the animation blueprint. Add Apply ACE Face Animations to the facial animation blueprint before the ARKit mapping node, then connect the output pose.
Configure audio invocation. Use Animate Character from Wav File Async, configure the WAV path and the appropriate A2F Provider, and set up an input event. The original tutorial uses the T key to trigger playback. Exposing parameters and switching models are optional extensions.

The NVIDIA official-plugin workflow used here targets UE. There is no equivalent ready-made official Unity plugin workflow that we could carry over directly for this tutorial. Using it in Unity would require additional integration for service or inference calls, facial-data mapping, and playback logic. We therefore did not produce a NVIDIA Unity integration tutorial for this comparison.
For projects involving both UE and Unity, or teams that want to validate an existing character using a ready-made sample, Mugen Morph's plugins and samples for both engines make it easier to get started. In this UE tutorial, menu-based character setup also reduces the steps needed to build the basic invocation chain manually. NVIDIA's workflow is better suited to developers willing to configure blueprints themselves who need local inference and more detailed parameter control.
Tutorials and documentation

Mugen Morph API and developer access: https://mugen3d.ai/api/
Mugen Morph Unity plugin and instructions: https://github.com/sumeurai/MugenMorph-For-Unity
NVIDIA ACE UE plugin documentation for the referenced version: https://docs.nvidia.com/ace/ace-unreal-plugin/2.5/ace-unreal-plugin-audio2face.html

6 Which integration steps require less work?

In our integration experience, Mugen Morph's convenience came mainly from common UE character settings being grouped in the plugin menu, together with an existing plugin and sample scenes for Unity. Developers can complete basic validation first, then integrate audio-driven animation into their own projects.

These differences make Mugen Morph a practical option for developers who want to get a sample working first and then integrate it into a project step by step. In UE, menu options reduce manual setup of the basic character configuration and invocation logic. In Unity, the sample supplies a complete workflow that developers can inspect and reuse.
For teams using both UE and Unity, having an existing integration path in each engine is a direct practical advantage of Mugen Morph. Characters that already have ARKit-compatible facial blendshapes can also be adapted from the sample, allowing teams to reuse existing assets.

7 Wait time before playback and usage costs

In this round, we used the same Audio2Face.wav file to compare the wait before playback in Mugen Morph's cloud workflow and NVIDIA's local LocalA2F-Claire workflow. Timing starts when the processing-request log appears in the screen recording and ends when speech first becomes audible. It excludes the earlier time spent browsing folders and selecting audio.

In this single test, the local Claire workflow had a shorter wait, by approximately 25.9 seconds. The times above are estimates from the original recordings, and each timestamp refers to its respective source video.
This measures the overall time from the processing request to audible speech. It includes workflow processing, playback preparation, and any silence at the beginning of the input audio. It is not the model's pure computation time or the latency to the first frame of facial animation.
Both recordings completed playback of the test audio, with no observed application exit or playback stopping partway through.

Mugen Morph pricing by audio duration

As recorded on October 13, 2026, Mugen Morph's developer pricing had no subscription or monthly fee and used tiered billing based on the amount of audio processed during the month. Usage tiers reset on the first day of each month, and usage beyond a tier's threshold moves into the next rate. Check the live developer dashboard and account terms before publication or use.

Monthly audio usage	Price
1–6,000 seconds	$0.002 per second
6,001–60,000 seconds	$0.0016 per second
60,001–600,000 seconds	$0.0012 per second
Above 600,000 seconds	$0.001 per second

At the first-tier rate, a 10-second clip costs approximately $0.02, and processing 3,600 seconds in a month costs approximately $7.20. With marginal tiered billing, 10,000 seconds would cost approximately $18.40: $12 for the first 6,000 seconds and $6.40 for the next 4,000 seconds. Budgeting should also account for retries, test clips, and repeated generations.

Hardware and licensing costs for NVIDIA's local workflow

NVIDIA's local workflow does not have the same per-second cloud API pricing as Mugen Morph. Official materials identify the Audio2Face-3D SDK's C++ and Python source code as MIT-licensed, while the model weights use the NVIDIA Open Model License. For product distribution, check the license files for the specific plugins, models, and dependencies you download.
For reference, the LocalA2F-James provider discussed in the hardware-cost information corresponds to the James v3.0 diffusion local provider. Official documentation lists a VRAM requirement of approximately 4.4 GiB or more, a supported NVIDIA Ampere, Ada, or Blackwell GPU, and the appropriate driver. Projects must also reserve VRAM for UE rendering, so being able to load the model does not guarantee that the target scene will run smoothly on the same device. This James hardware reference is separate from the Claire workflow used in the direct comparison above.

Cost factor	Mugen Morph cloud workflow	NVIDIA LocalA2F-James hardware reference
Direct invocation costs	Charged by processed audio duration	No equivalent per-second API fee for the local path
Client GPU	No specific NVIDIA GPU required for facial inference	Supported NVIDIA GPU required; James v3 needs approximately 4.4 GiB or more of VRAM
Network	An available connection is required at runtime	Local inference does not depend on a remote A2F service
Engineering time	Mainly plugin compilation, credentials, character mapping, and network-error handling	Mainly plugins and models, AnimBP, blueprints, hardware compatibility, and performance tuning
Operations	Provider maintains the server; client manages quotas and retries	Team manages target devices, drivers, model packages, and version compatibility
When the cost model is attractive	Predictable usage and a desire to reduce local deployment work	Hardware already meets requirements and local usage is sufficient to justify engineering effort

8 Which needs make Mugen Morph a good fit?

In this UE 5.6.1 test using the same MetaHuman character and default parameters, the two workflows had different strengths.

Dimension	Findings from this test
Installation and integration	Mugen Morph was more convenient in our hands-on setup; the local Claire workflow required more configuration
English lip sync	Both produced lip movements, with differences in opening amplitude and closure timing; no overall accuracy ranking was established
Facial expression	Mugen Morph had more active eyebrows, eyes, and mouth corners; Claire's upper face was more restrained
Wait before playback	In the latest single recorded run, approximately 28.7 seconds for Mugen Morph and 2.8 seconds for Claire
Playback behavior	Both completed playback of the test audio in this run

If your priority is getting a character connected in UE or Unity more quickly and you accept cloud calls, Mugen Morph is worth trying first. Its default output in this test also showed more noticeable movement around the eyes, eyebrows, and mouth corners.
If you prioritize local inference and are willing to invest in installation and configuration, LocalA2F-Claire is worth considering. In the latest test, it had a shorter wait from the processing request to speech playback.
There is no single winner across every dimension in this comparison. Integration convenience, lip-sync accuracy, expression style, and playback speed should be evaluated separately against your project's needs.
The direct comparison covers only one MetaHuman character, a limited set of audio samples, and default parameters. It does not represent all characters, speaking rates, or delivery styles. The NVIDIA workflow tested was local LocalA2F-Claire; the findings cannot be directly generalized to James, Mark, or cloud deployments. We did not score accuracy phoneme by phoneme, evaluate emotion recognition, quantify jitter, or test long-term stability.

9 Character adaptation and reuse of existing assets

For projects with existing character assets, a practical question is whether those characters can be reused for audio-driven animation and what adaptation is required.
The tutorials above show Mugen Morph integration paths in UE and Unity. Beyond MetaHuman, non-MetaHuman mesh characters with the required facial blendshape data can also use the relevant samples as a starting point, allowing existing characters to generate lip movements and expressions from audio.
This provides more options for reusing character assets. Actual integration still requires adaptation based on the character's facial blendshapes, parameter mapping, and plugin requirements. Developers can validate their own character with a short audio clip before integrating the full project workflow.

10 Development progress and future directions

Preview of the next Mugen Morph Mesh upgrade

Building on the current version, we are developing the next generation of our own Mugen Morph Mesh facial-animation system. The examples below show the new version's facial animation, with a focus on making lip movements, the eye and brow area, and overall expression work together more naturally and continuously during speech.
This demonstration uses two different characters and Chinese and English audio grouped into five emotions: Happy, Angry, Neutral, Fear, and Sad. Alongside the mouth movements, look at eyebrow raises and tightening, eye opening and closure, changes at the mouth corners, and the face as a whole to see how different emotions appear while the characters speak.
The overview image collects representative frames from the five emotion groups. The full videos show how the expressions change over the course of the speech. With this upgrade, we aim to help characters both follow the audio and convey more of the emotion in the delivery.

These are demonstrations of an unreleased development version. The earlier comparison uses the current plugin version.

Expanding to more types of characters

While continuing to improve facial animation for mesh characters, we also aim to gradually extend these capabilities to more character formats, including research into driving 3DGS characters.

Our goal is to let more kinds of character assets produce natural lip movements and facial expressions from speech, while fitting more easily into developers' existing creative workflows. We will continue sharing demonstrations and integration approaches as development progresses.

11 Frequently asked questions and developer access

Which is a better fit for UE developers, Mugen Morph or NVIDIA Audio2Face-3D?

If you want a quicker MetaHuman setup and accept cloud calls, try Mugen Morph first. If you want local inference and are willing to take on more configuration, consider NVIDIA LocalA2F-Claire. This article compares those two specific workflows.

Which has more accurate English lip sync?

We have not established an overall accuracy ranking in this round. Both produce opening and closing movements, but some movement amplitudes and timings differ. A wider mouth opening or richer expression does not automatically mean more accurate matching to the sounds being spoken.

Does more active facial expression mean better emotion recognition?

That cannot be concluded directly. The more visible eyebrow raises and lifted mouth corners in the UE videos are observable differences in expression. Emotion recognition was not separately evaluated in this test, and the five groups in the development demonstration do not constitute an emotion-recognition score.

Which workflow has a shorter wait before playback?

In the latest single test using Audio2Face.wav, Mugen Morph took approximately 28.7 seconds, and LocalA2F-Claire approximately 2.8 seconds. Timing starts when the processing-request log appears and ends when speech first becomes audible. These are wait times for the workflows in this test, not pure model computation times or a general performance ranking.

Is it fair to compare a cloud workflow with a local workflow?

It is possible to compare the integration steps, output, and wait times that developers experience, provided the deployment methods are explicit. Mugen Morph used the cloud here, while Claire ran locally. The results describe the overall workflows, and not every difference can be attributed to the models themselves.

Does easier integration mean that no build environment is needed?

No. The GitHub source-installation path used here involves Visual Studio and a C++ project. The convenience mainly concerns configuration, character setup, and the demo workflow after the plugin is enabled. For other installation methods, follow the instructions for that version.

Can the UE findings be applied directly to Unity?

No. The output comparisons and wait-time measurements in this round come from UE 5.6.1. Unity plugin integration, character adaptation, and runtime behavior need separate validation in the corresponding environment.

Can the development-preview capabilities already be used through the plugins?

The next-generation Mugen Morph Mesh version shown here has not yet been released. Use the current plugin version to judge what is available today. Exploration of driving 3DGS characters is a separate development direction; its integration should not be treated as already available through the current UE and Unity mesh-character workflows.
If you are choosing an audio-driven animation workflow, start with the existing plugin samples and use your own character and a typical audio clip to evaluate lip sync, facial style, and the setup experience.
Developer access: https://mugen3d.ai/api/

3D Gaussian Splatting vs. Meshes: Choosing a Representation for Real-Time Content

Mugen3D Dev — Fri, 30 Oct 2026 02:18:39 +0000

As real-time rendering and generative AI evolve, developers have more ways to represent visual content than traditional polygon meshes. One format attracting attention is 3D Gaussian Splatting (3DGS).

The interesting question is not whether 3DGS will replace meshes. It is where each representation fits in a production pipeline—and what trade-offs developers should consider.

Meshes and 3DGS represent scenes differently

A mesh describes a surface using vertices, edges, and polygons. This gives developers explicit geometry to work with. Meshes fit established workflows for UV mapping, material editing, collision, deformation, and skeletal animation.

3DGS represents a scene using many Gaussian primitives. Each splat has attributes such as position, color, opacity, and shape. Together, they can reproduce detailed appearance from different viewpoints, including visual features that are difficult to model as clean geometry.

That difference matters. A visually convincing representation is not automatically a production-ready game asset. A mesh provides explicit surfaces and topology; a 3DGS scene prioritizes appearance through a different representation.

Where 3DGS may be useful

3DGS is especially interesting for content with complex visual detail, such as hair, fur, foliage, and textured real-world environments. In reconstruction workflows, it can preserve the appearance of a captured scene without requiring every detail to be rebuilt as conventional geometry.

This could make it useful for certain environment elements, visual references, or scenes where appearance matters more than extensive deformation. But the right choice depends on the requirements of the project.

For example, a character that needs skeletal animation, precise collision, and frequent material changes may fit more naturally into a mesh-based pipeline. A captured environment that is mostly viewed from a limited range of angles may be a candidate for a splat-based representation. These are possibilities to evaluate, not universal rules.

Questions to ask before choosing a representation

When evaluating 3DGS or meshes for a real-time project, consider:

Does the asset need to deform? Characters and gameplay objects may need rigging, skinning, and predictable geometry.

Does it need collision or physics? Visual representation and collision representation may need to be handled separately.

How will the camera move? Viewpoint range can affect how a reconstructed asset looks.

How editable does it need to be? Consider whether artists need to change topology, materials, or individual scene elements.

What are the runtime constraints? Performance depends on the implementation, hardware, scene size, and rendering pipeline—not just the representation format.

Can the asset be reused elsewhere? A production pipeline may need exports for multiple engines, tools, or output formats.

A hybrid pipeline may be worth exploring

Meshes and 3DGS do not have to be treated as competing formats. A project could use meshes for characters, collision, and gameplay-driven objects, while using splats for selected environments or captured visual detail.

The challenge is integration. Teams need to consider import and export, runtime performance, lighting, interaction, level of detail, and how artists will inspect and edit the result. Those requirements will vary between engines and projects, so practical tests matter more than broad claims about one format being universally better.

3DGS makes a different point in the design space: it offers another way to represent detailed visual content. The useful question for game developers is where that representation solves a real production problem—and where the existing mesh workflow remains the better fit.

How are you evaluating 3DGS, meshes, or hybrid approaches in your own real-time projects? Which constraint has mattered most so far: editability, animation, collision, visual fidelity, or runtime performance?

References

Kerbl et al., “3D Gaussian Splatting for Real-Time Radiance Field Rendering,” SIGGRAPH 2023.

Mildenhall et al., “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis,” ECCV 2020.

This article was prepared with AI assistance and should be reviewed by the author for technical accuracy before publication.

From a Portrait to an Interactive TalkingHead: Persona, Knowledge, and Voice

Mugen3D Dev — Wed, 28 Oct 2026 06:15:22 +0000

I work on Mugen3D at SumeruAI. We have been exploring a workflow that turns a portrait into a character people can talk with through text or voice. This post walks through the creator flow and the pieces a developer needs to think about when building a product around it.

The character workflow

In TalkingHead Studio, the flow starts with a portrait and then adds the interaction layer:

Upload a portrait to define the character's appearance.
Write a persona with the character's background, tone, and conversational goal.
Add a knowledge document for topic-specific reference material.
Choose a voice.
Start a text or microphone conversation and review how the character responds.

The persona and knowledge document serve different purposes. The persona shapes how the character communicates; the document supplies reference material for what it can discuss. A history guide might use curated sources, while a product guide might use FAQs. In either case, the material should be organized and the answers should be tested. A document upload does not guarantee factual responses.

Voice and interaction design

Text is useful for quick prompts and review. Voice can make the character feel more conversational, but it adds practical questions: how long does a response take, how does the UI show that it is listening or speaking, and what happens when speech recognition or playback fails?

Voice, persona, and knowledge content should be tested together. The same answer can feel quite different depending on the voice and speaking animation. For a useful prototype, I would start with one clear role and a small set of reference material, then test whether people can understand what the character can and cannot do.

What the developer integration covers

There are two related parts: creating a TalkingHead asset from a photo, and embedding a rendered character in your own product. The Mugen3D API page describes photo-to-TalkingHead generation as an asynchronous job with a result callback. The public API docs are the source for the current request fields and job details.

For the browser runtime, the TalkingHead Web SDK repository includes a demo, a minimal integration example, avatar rendering helpers, local workers, and audio-driven facial animation. Its example flow is to get an access token, load a hosted avatar model, and pass a WAV file to the audio-driven facial animation helper. The SDK covers avatar rendering and facial drive; your application still needs its own conversation interface and any retrieval or product logic unless you choose a service that provides those pieces.

Keep API credentials on your server. For a longer-running generation task, keep the task ID, show processing, completed, and failed states, and preserve useful error details so the user can recover after refreshing the page. The repository's minimal example and developer guide show the current SDK calls.

Use cases and limits

A small team could prototype a language-practice partner, a history explainer, an interview coach, or an interactive story character. The useful product work is often in the surrounding flow: how users choose a character, what they can ask, how the app handles uncertainty, and when it hands off to a human or a static answer.

For fan experiences or memorial projects, use portraits, recordings, and other source material only when you have permission. Make the synthetic nature of the character clear, and respect the wishes and privacy of the people involved. Voice cloning needs particular care.

Try it and share feedback

I would be interested in feedback from developers building conversational characters: which part is hardest to get right in a first prototype—consistent persona, useful knowledge references, voice interaction, or integration?

More prototype screenshots

The Boundary Between Video and 3D Is Blurring: Atlas, Genie 3 and Reusable Assets

Mugen3D Dev — Wed, 21 Oct 2026 08:51:35 +0000

Disclosure: I am part of the Mugen3D team at Sumeru AI. This article compares product approaches, not benchmark results. AI assisted with adaptation and editing. Product specifications and availability below reflect the cited documentation as of September 2026.

On September 1, 2026, World Labs introduced Atlas. Earlier, Google DeepMind’s Genie 3 had already allowed users to enter worlds generated in real time by a model.

Taken together, these two developments suggest that the industry is beginning to ask a different question.

The focus is shifting from:

Can the video look more realistic?

to:

Can visual content become a world that users can enter, change, and reuse?

This also explains why video and 3D are increasingly being discussed together.

Video is good at presentation. 3D is good at preserving spatial structure and interaction.

Atlas starts from spatial intelligence. Genie 3 starts from real-time world generation. Mugen3D at Sumeru AI starts with reusable 3D assets.

The three approaches are related, but they operate at different product layers and solve different problems.

Genie 3: The model generates what comes next as the camera moves forward. Its focus is real-time exploration and world simulation.

Atlas: The model understands images, videos, camera positions, and spatial relationships before generating new views, videos, or 3D outputs. Its focus is spatial control and reconstruction.

Mugen3D: The system first turns people, objects, and environments into reusable 3D assets, then enables animation, dialogue, and interaction in real-time applications. Its focus is asset production and application deployment.

Video Is About More Than Image Quality

Before generative AI, video and 3D followed two very different production paths

3D meant high cost and a high technical barrier. Its strengths were free camera movement, real-time interaction, and asset reuse. It was mainly used in games, visual effects, film production, and industrial design.

Video was treated as the default format for realistic content. It was easier to create and distribute, but largely “one-time” content. Once rendered, it was difficult to change the camera, edit individual objects, or reuse the scene as an interactive environment.

Generative AI first disrupted the video industry.

Video-generation models can now produce highly realistic footage. However, several limitations remain:

long-term consistency is still difficult;
commercial generation length is often limited;
users have limited control over camera movement;
training and inference remain computationally expensive;
video content still cannot be interacted with in the same way as a real-time 3D environment.

The core problem is this:

Video models generate fixed visual sequences. They do not automatically generate living, editable, and reusable spaces.

If video itself becomes three-dimensional

If the long-term goal is to generate high-quality visual content that is:

not limited by video duration;
freely navigable;
highly detailed;
interactive in real time;

then there are two broad paths.

Starting from video generation

A video-first system needs to solve several difficult problems at the same time:

maintaining consistency under diffusion or transformer-based generation;
achieving high-quality, low-latency interaction;
allowing users to control the camera;
generating long-duration content;
compressing and transmitting the result under real-world network conditions.

Starting from 3D generation

A 3D-first system can avoid some of these problems because a 3D asset is a reusable representation.

It naturally provides:

better spatial consistency;
longer-duration content;
user-controlled camera movement;
asset reuse;
easier integration with real-time interaction.

It also introduces a different set of challenges:

the quality of generated 3D assets must approach the visual quality of video;
animation generation is more difficult;
AI-driven 3D motion and behavior are still evolving;
deployment requires careful handling of memory, compression, rendering, and interaction.

The key difference is that the 3D path may exchange some generation complexity for greater engineering predictability.

The video path must solve several independent problems at once. The 3D path concentrates more of its difficulty in asset quality, animation, and runtime deployment.

At present, three technical approaches are helping validate this direction.

Atlas: Bringing 3D spatial understanding into video generation

World Labs introduced Atlas as an omni world model for spatial intelligence.

According to the official description, Atlas operates natively on text, images, videos, and 3D data. It combines these inputs into a shared spatial context and uses that context to generate new views, videos, reconstructions, and simulations.

Atlas supports several important capabilities:

Taking one to six input images and generating video with manually designed camera paths, up to one minute at 1440p.
Reconstructing real-world scenes from one to dozens of input images and producing novel views as well as explicit 3D outputs.
Modeling space and time from input videos for applications such as visual effects and real-to-sim workflows.

Atlas also supports image generation and 360-degree panoramas.

Its main value is that it attempts to understand spatial relationships before generating the next image or video frame. It is not simply predicting pixels independently. The model uses information about camera position, geometry, depth, and the surrounding scene.

However, there is still an important distinction between spatially aware video generation and a fully interactive 3D runtime environment.

A camera-controlled video and an explicit 3D reconstruction are different output modes. In a video workflow, the timeline remains the primary structure. In a 3D application, the scene can be rendered repeatedly from different viewpoints while individual assets remain available for further editing or interaction.

Atlas significantly improves spatial control and reconstruction, but its product workflow should not automatically be treated as the same thing as a reusable 3D asset pipeline.

Genie 3: A generated world that responds to movement

Google DeepMind introduced Genie 3 as a general-purpose world model that can generate interactive environments.

According to Google DeepMind, Genie 3 can generate dynamic worlds from text prompts and allow users to navigate them in real time at 24 frames per second and 720p, with consistency maintained for a few minutes.

The breakthrough comes from the generation loop.

In a traditional video:

the video is generated first;
the user can only watch it afterward.

In Genie 3:

the user moves or changes direction;
the model predicts what should appear next;
the system generates the next part of the world;
the environment remains broadly consistent with the previous trajectory.

Genie 3 also supports promptable world events. Users can change weather conditions or introduce new objects and characters.

This means that the user’s movement becomes part of the generation process.

For robotics and embodied-agent research, this creates a new type of simulated environment. An agent can move through a generated world, test actions, and observe how the environment evolves.

However, the current limitations are also clear. Google DeepMind lists the following constraints:

the direct action space is still limited;
complex interactions between multiple independent agents remain difficult;
real-world locations cannot yet be reproduced with perfect geographic accuracy;
text rendering is not always reliable;
continuous interaction currently lasts only a few minutes.

From a product perspective, Genie 3 is best understood as a real-time generative world model.

It allows users to enter a generated environment, but it does not automatically provide a library of independent meshes, 3D Gaussian Splatting assets, animation rigs, or objects that can be exported directly into Unity or Unreal Engine.

That is a different problem from world generation.

Project Genie

In January 2026, Google introduced Project Genie, an experimental web prototype powered by Genie 3, Nano Banana Pro, and Gemini.

Project Genie allows users to create, explore, and remix interactive worlds using text, images, and other inputs.

However, Project Genie is not identical to the full Genie 3 research model.

Google’s current documentation lists several limitations:

each generation is limited to 60 seconds;
generated worlds may not always follow prompts, images, or real-world physics closely;
characters can be difficult to control;
some Genie 3 capabilities, including promptable world events, are not yet included in the prototype.

Genie 3 represents the research capability.

Project Genie represents the product experience currently available to a limited audience.

Mugen3D: Turning the world into reusable 3D assets

Product and public demos: Mugen3D.

Mugen3D follows a different path.

Instead of generating every frame during interaction, Mugen3D starts by creating a reusable 3D representation. Animation, dialogue, and interaction are then built around that representation.

The overall workflow can be summarized in three steps:

Generate 3D Gaussian Splatting assets from a single image.
Generate 3D environments from sparse images or videos.
Use AI-driven animation and real-time rendering to create interactive experiences.

The generated content is no longer limited by the duration of a video file. Once the 3D asset exists, it can be rendered repeatedly from different viewpoints and used in different application contexts.

Mugen3D’s current product direction includes 3DGS scenes, realistic characters and animals, stylized assets, and interactive AI avatars.

Our development work also covers AI-driven animation for 3D virtual humans, including facial expressions, micro-expressions, motion, dialogue, and real-time interaction.

At the same time, generated 3D assets should not be treated as automatically production-ready in every scenario.

Single-view reconstruction still has clear limitations:

unseen surfaces must be inferred;
small text and logos may be inconsistent;
reflective and transparent materials are difficult;
complex geometry may require manual review;
output quality depends heavily on the input image and the intended use case.

For this reason, Mugen3D should currently be evaluated as a fast 3D asset-production and visualization workflow. It is not a universal replacement for CAD or manual modeling.

The differences are mainly about cost, interaction, and application

Dimension	Atlas	Genie 3	Mugen3D
Main workflow	Spatial generation, camera-controlled video and reconstruction	Real-time interactive world generation	Reusable 3D asset generation and application integration
Outputs and reuse	Depends on the selected mode; includes explicit 3D reconstruction	Navigable generated experience; do not assume independent asset export	3D representations intended for repeated rendering and integration
Main engineering questions	Input coverage, camera control and output-mode requirements	Temporal consistency, action limits, latency and inference	Unseen surfaces, asset quality, animation, memory, compression and rendering
Cost comparison	Depends on workload and output mode	Depends on duration, resolution and interaction	Generation plus runtime rendering and animation; no standardized cross-system benchmark claimed

This workflow comparison replaces the original infographic. It does not establish relative training cost, image quality or throughput.

The fundamental difference in cost structure

Atlas and Genie 3 both rely heavily on generative inference.

For video or world-model generation, computation increases with output duration, resolution, and interaction time. The system must continue producing new visual content as the user moves or as the timeline progresses.

Mugen3D shifts more of the work into the asset-generation stage.

Once the asset has been generated, the runtime application can render it repeatedly without regenerating every frame. The remaining costs are related to rendering, animation, memory, compression, and deployment.

The difference in interaction

Genie 3 allows users to navigate through a generated environment. The world responds to movement, but the underlying content is still produced as part of the model’s generation loop.

Atlas provides camera-controlled generation, novel views, and spatial reconstruction. Its output depends on the selected workflow and output mode.

Mugen3D is designed around reusable 3D assets. In an application, these assets can be positioned, rendered, animated, and combined with other systems.

This is the difference between:

watching generated content;
navigating generated content;
building an application around reusable 3D content.

The difference in commercialization

Genie 3 remains a limited research preview.

Project Genie is an experimental prototype available to a limited group of Google AI Ultra subscribers in the United States.

Atlas is entering early access with selected partners.

Mugen3D is following a public product and demo path, with ongoing development and product releases.

This does not mean that one approach is automatically better. It means the three technologies are currently at different stages of productization and are designed for different deployment scenarios.

The difference in application scope

Genie 3 and Atlas are particularly relevant to:

interactive world generation;
robotics and embodied-agent research;
simulation;
camera-controlled visual content;
spatial reconstruction.

Mugen3D is designed for an asset-oriented workflow, including:

interactive 3D assets;
digital humans;
virtual characters;
real-time applications;
virtual showrooms;
simulation;
interactive media.

The central question is whether the user only needs to experience the generated world, or whether the user needs to reuse and integrate the underlying assets.

Video may eventually become content that users can enter

Atlas and Genie 3 have pushed the industry’s imagination forward.

Atlas makes generative systems more spatially aware. It can control camera movement, reconstruct scenes, and produce 3D-related outputs.

Genie 3 connects movement, time, and generation. Users can enter a generated environment and influence what appears next.

Mugen3D follows an asset-oriented application path.

It first turns key objects and environments into reusable 3D assets, then uses animation, dialogue, and real-time rendering to make them move and interact inside a 3D space.

Its potential advantage comes from changing the unit of generation.

Instead of generating only a sequence of frames, the system attempts to generate a reusable representation that can support multiple viewpoints, interactions, and application workflows.

The future of screen-based content may not be limited to video or games.

People may continue to watch stories as videos, enter generated environments as interactive worlds, and communicate with digital characters in real time.

Robots may also train in simulated 3D environments, while the same underlying content is reused for education, entertainment, commerce, and social interaction.

The user may be looking at the same three-dimensional content, but choose to:

observe it;
explore it;
edit it;
or participate in it.

The companies and technologies that can turn research into usable products, bring those products to real users, and make people return to the experience may be closer to the next generation of visual computing.

The boundary between video and 3D is not disappearing because every video will become a 3D model.

It is disappearing because users increasingly expect visual content to be:

explorable;
editable;
interactive;
and reusable.

Sources and notes

The dates, specifications, and product statuses of Atlas, Genie 3, and Project Genie are based on information publicly available on their respective official pages as of September 2026.

Official sources:

Atlas: https://www.worldlabs.ai/blog/atlas
Genie 3: https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/
Project Genie: https://blog.google/innovation-and-ai/models-and-research/google-deepmind/project-genie/
Mugen3D: https://mugen3d.ai/

Discussion question:

In your workflow, what matters most: visual fidelity, camera control, asset reuse, real-time interaction, or predictable runtime cost?

Testing Single-Image 3DGS for E-commerce: From One Product Photo to an Interactive Web Asset

Mugen3D Dev — Fri, 16 Oct 2026 02:53:09 +0000

Traditional product photos and turntable videos show only predefined views. Interactive 3D lets shoppers rotate a product and inspect its sides, back, and overall volume.

Creating a 3D scan or manual model for every SKU is costly. This article tests a lighter approach: reusing an existing product photo to generate a free-viewpoint 3DGS asset.

Using SumeruAI's Mugen3D, we focus on the actual workflow, differences between product categories, and the quality checks required before batch deployment.

Note: API capabilities reflect current SumeruAI website and console information. The e-commerce integration architecture below is a suggested pattern; actual fields and outputs should be verified against current documentation and API responses.

1. The Gap Between Fixed Product Photos and Interactive 3D

Product photos and turntable videos help show appearance, but shoppers still cannot rotate the product or inspect any angle on demand.

For a product detail page, the core interaction is simple:

Rotate the product freely;
Inspect its sides, back, and overall volume;
Embed it in product pages, digital catalogs, or virtual showrooms;
Reuse existing product images and reduce additional 3D capture work.

Here, “interactive 3D for e-commerce” means orbiting, previewing, and embedding—not CAD data, manufacturing files, or physical simulation.

2. What Does Single-Image Generation Produce?

Mugen3D's Image to 3DGS feature accepts one product image and generates a previewable, exportable 3DGS asset as an asynchronous task. The website offers SQ and HQ quality modes; current output formats and download methods should be confirmed in the console documentation.

3DGS, meshes, and CAD serve different purposes. This article evaluates the output as a product-visualization asset.

A single image shows only one viewpoint, so the system must infer the back, bottom, and occluded structures. A rotatable result does not mean every unseen surface was measured accurately.

The output is best used for visualization, not assumed to be an engineering-accurate model.

Test Products and Selection Criteria

The test covers six categories representing common challenges: hard surfaces, soft materials, mixed materials, functional structures, transparency and reflections, and fine details.

3.1 Robot or Hard-Surface Toy

Robots and hard-surface toys have clear silhouettes and distinct components. The main checks are proportions, component connections, and perceived volume.

3.2 Plush Toy

Plush toys have soft surfaces and irregular silhouettes. The main checks are overall shape, fur appearance, and whether the inferred back looks natural.

3.3 Designer Toy or Collectible Figure

Designer toys combine exaggerated proportions, decorative details, and sometimes mixed materials. The main checks are character identity, material transitions, and consistency of the back view.

3.4 Shoe

Shoes combine curved surfaces, soles, openings, and hidden side structures. The main checks are functional structure, left-right relationships, and inferred geometry.

3.5 Cosmetic Bottle

Cosmetic bottles often include transparent or semi-transparent materials, reflections, labels, logos, and small text. The main checks are silhouette, label stability, and material appearance.

3.6 Watch

Watches combine glass, metallic reflections, dial markings, logos, and small text. The main checks are dial structure, strap connections, and detail stability.

Based on this test, robots or hard-surface toys, plush toys, and designer toys produced more stable silhouettes and materials, making them good candidates for initial batch generation. Shoes may be used at scale with sample-based human review. Cosmetic bottles and watches are more sensitive to transparency, reflections, small text, and complex details; for now, they are better suited to individual tests and manual review rather than automatic batch publication. For batch input, use consistent images with a complete product, clear silhouette, simple background, even lighting, and no severe occlusion.

4. From a Product Image to a 3DGS Asset

The Studio workflow is simple:

Upload a product image and confirm the subject;
Select a quality mode and click Generate;
Let the task run in the background for about 30 minutes while you continue other work;
Return to preview, rotate the result, and export the asset.

For batch testing, record the quality mode, actual generation time, file size, and review result to identify categories suitable for scale.

5. Generation Flow Based on Public API Information

Mugen3D's public API uses asynchronous jobs: the server uploads an image with a Bearer token, receives a task ID, then uses that ID to query status and retrieve the result.

The developer workflow has four steps:

Submit image → Receive task ID → Query task status → Retrieve the 3DGS asset

SQ submission example:

import requests


def submit_mugen_sq(access_token: str, image_path: str):
    url = "https://api.sumeruai.us/v1/mugen-3d/sq"
    headers = {"Authorization": f"Bearer {access_token}"}

    with open(image_path, "rb") as image_file:
        files = {"file": image_file}
        response = requests.post(
            url,
            headers=headers,
            files=files,
            timeout=120,
        )

    response.raise_for_status()
    return response.json()


result = submit_mugen_sq(
    "YOUR_ACCESS_TOKEN",
    "product.jpg",
)

task_id = result["data"]["id"]
print("Task ID:", task_id)

Before publishing an integration guide, verify authentication, task-status, download, and HQ request details against current console documentation and actual API responses.

6. A Proposed E-commerce Integration Architecture

Mugen3D generates the 3DGS asset; object storage, CMS, SKU binding, and the web viewer remain part of the merchant's existing system. After generation and review, the delivery path can be simplified to:

Approved 3DGS asset → Object storage or CDN → CMS SKU binding → Web viewer → Product detail page

Display the 3D viewer only when the asset has passed review and loads correctly; otherwise, fall back to the existing product images.

The CMS could store:

Product SKU;
Generation task ID;
Current task status;
3D asset location;
Generation version;
Review status;
Failure reason or internal notes.

This is a suggested architecture. Mugen3D does not provide a complete CMS or product detail page system.

7. How to Evaluate Results Across Product Categories

To avoid presenting only the most successful angle, every product should be reviewed using the same criteria:

At minimum, review the following:

Does the front remain consistent with the input image?
Are the left and right sides structurally plausible?
Do the back and bottom contain obvious artifacts?
Have the product proportions changed?
Are logos, labels, and small text legible and stable?
Are any components missing or duplicated?
Do transparent, reflective, or soft materials look natural?
Do floating elements, breakage, or visible noise appear during rotation?

The results can then be grouped into three levels:

Ready for the presentation workflow: Major viewpoints are stable, with no obvious silhouette or structural errors;
Manual review required: The overall result is usable, but text, logos, the inferred back, or materials present a risk;
Not currently recommended: The product is incomplete, structurally inaccurate, or substantially different from the real item.

8. Production Considerations

Before moving to batch use, add the following safeguards:

Keep credentials on the server

Store access tokens only on the server. The front end should receive only approved asset URLs.

Validate images before submission

Validate file type, size, resolution, and subject quality before submission. Send images with complex backgrounds, severe occlusion, or blur for manual preparation.

Add timeout, polling, retry, and error handling

Store the task ID and use a polling interval, timeout, limited retries, and error logging.

Generation success does not mean publication approval

Use category-specific rules for automatic approval, manual review, or unsupported products instead of publishing every completed result automatically.

Test mobile performance separately

Test asset size, first load, viewer compatibility, memory use, and failure fallback on real mobile devices and representative networks.

9. Suitable and Unsuitable Use Cases

Single-image 3DGS may be useful as a visual supplement for:

360-degree product exploration;
E-commerce product detail pages;
Digital product catalogs;
Virtual showrooms;
Marketing and social media content;
Visual previews in games or applications;
Concept demonstrations and rapid prototypes.

Without additional validation, it should not be treated as a direct replacement for:

CAD design or precise measurement;
Industrial manufacturing and tooling;
Use cases that require exact small text or logo reproduction;
Products that demand highly accurate transparent, glass, or mirror-like materials;
Workflows that require rigging, collision geometry, or editable topology;
AR try-on experiences with strict dimensional and occlusion requirements.

10. Conclusion

The value of single-image 3DGS is not to replace product photography or 3D modeling, but to turn existing product images into free-viewpoint visualization assets more efficiently.

Based on this round of results, robots or hard-surface toys, plush toys, and designer toys or collectible figures are suitable for initial batch generation. Shoes may be used at scale with sample-based human review. Cosmetic bottles and watches are not currently recommended. Any decision to publish these assets should also consider file size, mobile performance, and the business’s tolerance for visual errors.

Next, we will continue testing web-based 3DGS viewers for compatibility, compression, mobile loading, and catalog-scale SKU management.

Has anyone tested Gaussian Splatting viewers on production e-commerce pages? I’d be interested to hear how others handle viewer performance, file compression, and mobile loading optimization.

Same model, same audio: comparing 3 Audio-to-Face approaches in Unity

Mugen3D Dev — Thu, 20 Aug 2026 12:16:51 +0000

We’ve been testing different Audio-to-Face approaches for real-time characters, and one thing became obvious very quickly:

Even with the same character and the same source audio, different systems can produce noticeably different facial animation.

So for this test, we compared three approaches under the closest conditions we could reproduce:

Mugen3D Audio To Face
NVIDIA Audio2Face
Convai

For this comparison, we mainly looked at three things:

Timing— how closely the mouth movement follows the speech
Mouth movement — how restrained or exaggerated the articulation feels
Overall naturalness — whether the face feels believable while speaking

What we observed

NVIDIA Audio2Face

NVIDIA provides a flexible and technically mature solution, especially for developers who want more control over the overall pipeline.

In our test, the facial movement felt relatively clear and structured, although bringing the workflow into Unity requires more integration work.

Convai

Convai produced noticeably larger mouth and facial movements in our test.

The result feels more expressive and performance-oriented, which can work well when combined with full-body character animation.

Mugen3D

Our current approach is more restrained.

We’re focusing on keeping the mouth movement closely aligned with the source audio without making every syllable feel overly exaggerated, while keeping the Unity integration lightweight.

A separate reference: Georgy Dev

We also looked at Georgy Dev’s Audio-to-Face result.

However, we did not have access to the same character setup, so we don’t think it would be fair to include it in the direct comparison above.

Visually, the result looked very polished in the material we reviewed, so we’re including it here only as an additional reference rather than treating it as an apples-to-apples benchmark.

That distinction is important to us: different character rigs, rendering setups and production pipelines can significantly affect how facial animation looks.

What matters most to you?

When evaluating Audio-to-Face for a real project, which matters more?

tighter timing with the source audio
more expressive facial movement
overall visual naturalness
easier Unity integration

We’d be interested to hear how other Unity developers judge these results.

If you want to try our current Unity version yourself, the demo is available on GitHub:

[GitHub link]：https://github.com/sumeurai/AudioToFace-For-Unity