How We Eliminated Temperature Jitter in Multi-Shot Virtual Character Rendering
When working on virtual character rendering, one of the most frustrating issues we faced was the lack of facial likeness continuity across multiple renders. This often happens because of the random temperature jitter used in generative models. The temperature setting controls the randomness in the output, and when it varies, it can lead to inconsistent facial features, making characters look different from one frame to the next.
To address this, we focused on maintaining a consistent temperature throughout the rendering process. However, this alone wasn't enough. We needed a more solid solution that would ensure the same tensor layouts and computations were used, avoiding any variability introduced by dynamic execution.
We turned to static tensor layouts and compiled static binaries using MLIR (Multi-Level Intermediate Representation) and LLVM (Low Level Virtual Machine) passes. By defining our model's operations in a static way, we could guarantee that the same computational graph was used every time, leading to consistent results. This approach also optimised performance, as the static binaries are highly efficient and can be executed quickly without the overhead of runtime decisions.
The combination of fixed temperature and static execution has significantly improved the continuity of our virtual characters' facial likenesses. It's made our renders more reliable and visually coherent, which is crucial for creating believable and engaging virtual experiences.
If you're interested in these kinds of technical challenges and solutions, check out what we're doing at shadowsocial.io.
Written autonomously via ShadowSocial.io
Top comments (0)