In this article I’ll describe what issues we’ve encountered during development of our latest Reunion live wallpaper and its WebGL demo.
All gorgeous hand-painted artwork is done by Conrad Justin, with only minor additions by me and my brother (lights for candles, dust particles, etc). You can compare our scene with an original 3D model on Sketchfab here and also make sure you take a look at the artist’s full portfolio here.
Usage of FP16 textures
In both web demo and Android app we have animated objects for characters and decorations. And for this new art style of wallpaper scene, we introduced new types of rendering techniques and custom shaders. Here we have used vertex animations which are stored in floating-point textures (so the web demo requires WebGL 2).
For our use case precision of half-float to store vertex positions is just good enough. And according to OpenGL ES 3.0 specs, FP16 textures even can be filtered, which is really handy for animations - linear interpolation between animation frames will be handled by hardware virtually for free, without calculations in shader.
Issues with textures filtering
Well, using texture filtering for interpolating animations was a good option only in theory - all you have to do is enable GL_LINEAR for texture and you’re good to go. However in practice, the arithmetic precision of filtering is not perfect and is even somewhat hardware-specific.
During development on PC everything was fine, but testing on different mobile phones with various GPUs revealed noticeable visual differences in hardware FP16 texture filtering. I suspect that the ANGLE wrapper for WebGL always uses full-precision floating point values in both shaders and textures, because we’ve already encountered precision issues in shader calculations during development of the Iceland WebGL demo - they were seen only on mobile devices with native WebGL-to-OpenGL ES translation.
Here is the reference rendering of squirrel character on PC without issues (Windows 10, Chrome or Firefox with ANGLE renderer):
And here are some results from different phones:
Galaxy A21S with Mali GPU. Vertex shivering with some gaps between triangles:
Moto E6s with PowerVR GPU. This one is closest to reference rendering on PC, but still has some gaps:
And Google Pixel 3 with Adreno 630 GPU. Shivering is also present:
Solution
The most obvious fix would be to use 32-bit GL_FLOAT textures which have better precision and should be interpolated more correctly, but unfortunately they are not filterable at all and animations look like in Quake 1 - without any interpolation:
So we decided to implement a custom linear interpolation in the vertex shader. In our implementation, textures represent animation in the direction of increasing Y coordinate, so we have to sample only two adjacent texels in this direction.
You can examine the source code of this shader here. As you can see, it uses highp precision for floats for precise and smooth interpolation of vertex positions.
First, this shader has a function getCenter() which returns a center of texel for any arbitrary coordinate. It is used to get the color of two points.
Actual filtering is done in the linearFilter() function, which samples two values half-texel higher and half-texel lower and linearly interpolates them based on how far centers of their texels are from actual sampled coordinates.
Please note that shader samples colors not exactly half-texel higher and lower but a texel height is multiplied by a value slightly smaller than 0.5 - 0.49. This is done because floating point precision is limited and sampling exactly at the edge of 2 texels might get into texels which we don’t need. This will result in a broken animation - interpolation with the previous or next frame, instead of the current one. Sampling at offsets slightly lower than half texel height eliminates this issue.
This implementation of custom texture filtering results in smooth interpolation, and it is identical on all tested platforms.
Result
You can take a look at live demo page with this custom interpolation here - https://keaukraine.github.io/webgl-reunion/
And of course full source code is available on Github - https://github.com/keaukraine/webgl-reunion
Oldest comments (0)