Interpolating Between 2 Aspect Ratios in a Slightly Overcomplicated Way

The other day, I had to work on a simple Lerp calculation for a Unity project. It was about lerping (= Linear Interpolation) an offset value from 3.0f to 4.0f, depending on the current window aspect ratio.

The initial specifications from the director were quite straightforward:

• In 'Vertical Mode', when the game's window is narrow, the offset should be at the lower end (3.0f).
• In 'Horizontal Mode', when the window is wide, the offset should be at the upper end (4.0f).
• To prevent the offset from becoming too extreme, the value should be capped at each end:
  • If the aspect ratio becomes smaller than 1080 * 1920 (a 9 : 16 aspect ratio), the offset should be capped at 3.0f, and should not be smaller than the cap.
  • If the ratio exceeds 1920 * 1080 (a 16 : 9 ratio), the offset should be capped at 4.0f. No greater than that.
  • When the aspect ratio is 1:1, the offset value should be at 50% of the range, i.e., (3.0 + 4.0) / 2 = 3.5.

By the way, users can freely adjust the window size to whatever they want, hence the need for capping.

The image above summarizes and visualizes the details:

1. Each capping end and the middle screen resolutions are denoted by the yellow boxes and the yellow 'resolution' points.
2. Since the game's window can be any size, any resolution points on the quadrant should be properly mapped to a lerp factor. (0.0f to 1.0f)
   • If a resolution point lies along the y = x line, such as (800 * 800), the factor must be 0.5f.
   • If a resolution point lies on or below the y = 1080 / 1920 x line, such as (480 * 270), the factor should be 0.0f.
   • If it lies on or above the y = 1920 / 1080 x line, such as (100 * 1000), the factor should be 1.0f.

1. Can't just plug in the Aspect Ratio itself.

Simply plugging in the aspect ratio to obtain the lerpFactor wouldn't suffice. While it nicely interpolates between the 2 ends, it wouldn't yield the middle value when the aspect ratio is 1:1.

var minRatio = 1080/1920; // = 0.5625f
var maxRatio = 1920/1080; // = 1.7778f

// Capping the ratio
var aspectRatioClamped = Mathf.Clamp(aspectRatio, minRatio, maxRatio);

// Getting the lerpFactor (0 - 1)
var lerpFactor =  aspectRatioClamped - minRatio / (maxRatio - minRatio);

// NOTE: When the ratio is 1:1, the lerpFactor shall NOT be 0.5f. 

When the ratio is 1:1, the lerp factor would be 1 - 0.5625 / (1.7778 - 0.5625) = 0.5372, slightly off from exactly 0.5f.

Converting the ratio into numerical form means setting the denominator to 1. In the graphical representation below, it's like obtaining the y-value of the intersection point between the line x = 1, and y = ratio * x.

For instance, if I am trying to get the lerp factor of a resolution of 960 * 640, the aspect ratio would be 960 / 640 = 1.5. To derive the lerp factor, I should first find the intersection point between x = 1, and y = 1.5 x, which is (1, 1.5). Then, I can perform the inverse-lerp with the y-value, to get a lerp factor of (1.5 - 0.5625) / (1.7778 - 0.5625) = 0.7714.
Any resolution proportional to this would have the same aspect ratio of 1.5, regardless of the size, thus yielding the same Lerp factor.

The x = 1 line vertically slices the ratio graph; as it's not perpendicular to each capping end, it naturally divides the range where slope < 1 and slope > 1 unevenly.

It's basically like performing an off-center projection (= oblique projection), so the center happens to be slightly off.

2. The Atan() way

To circumvent the unevenly divided range situation, one of the simplest approaches is to calculate the angles directly.

• Get the angle between the resolution point to the x-axis (referred to as theta here).
• Compare and perform an inverse-lerp operation on the angle within the range of (alpha to beta)
• This ensures that the ranges around the midpoint are evenly divided; a resolution of 1:1 would yield a lerp factor of 0.5f.

FYI: (1920/1080 = 30 degrees, 1080/1920 = 60 degrees)

To represent this in C# code, it would appear as below:

private float AtanLerp(Vector2 resolutionPoint)
{
    var resolutionPointAngle = Mathf.Atan2(resolutionPoint.y, resolutionPoint.x);
    resolutionPointAngle = Mathf.Clamp(resolutionPointAngle, MinAngle, MaxAngle);

    float lerpFactor = (resolutionPointAngle - MinAngle) / (MaxAngle - MinAngle);
    return lerpFactor;
}

The code directly calculates the angle between the resolutionPoint to the x-axis, then checks whether the angle lies within the angular range of the 2 capped aspect ratios.

There's no need for projection to a vertical slice line; it works as intended. This was the approach I initially took in the codebase.

3. The Projection Matrix Way

After a couple of weeks from the implementation, I happened to glance over the graph I scribbled while rifling through the notebook. The graphs had somehow etched into my brain, and then I realized a function call to atan2() for each frame could be somewhat costly.

In reality, the performance cost of the function is virtually negligible. There are plenty of other parts that impose much heavier workloads. Nevertheless, I decided to delve into this during my free time for no reason.

The major issue with the 1st method (using the aspect ratio) is that the 'projection line' wasn't aligned with the projection range. As long as I set the 'projection line' (the green one) perpendicular to the mid-range slope (the dark-blue line: y = x), evenly dividing the projection range (the light-blue lines), there will be no more skewed projection issue.

Once I obtain the projected point (the yellow point where the green and yellow lines intersect) from a resolution point, I can then calculate the lerp factor with a few floating-point arithmetic operations by comparing the projected point with the capping points (the green points).

This approach sounded quite promising. The projection could be handled with a matrix-vector multiplication, which could potentially be SIMD accelerated. Additionally, a few floating-point additions, subtractions, and divisions wouldn't hurt the performance as much as atan2() would.

3.1. Projection Matrix

I can derive a projection matrix that transforms a resolution point into a 'projection point', utilizing the very same technique that every game engine uses, but in a way much simpler way.

A 'projected point' is simply the intersection point between the projection line and the line that connects the origin point and a resolution point.

The Latter, the line connecting the origin and the resolution point, could be represented as P(t) = t * P, where P is the resolution point, and t is a scalar parameter.

To get the intersection point (= projected point), plug in the parameterized P(t) to the projection line y = -x + 2, and then rearrange the equations to solve for t.

After rearranging the equation, it becomes clear how to determine the projected point from the resolution point P. For each element of P, multiply it by 2, then divide it by (y + x).

The matrix below will perform such a calculation with a homogeneous 2D vector (augmented with w = 1).

After homogeneous division, I can finally attain the projected point.

3.2. Calculating the Lerp Factor

Locating the projected point (x, y) is one thing, but I still need to derive the lerp factor from it.

To put it naively, I could compare the length between the projected point to a min or max projected point with the length between the min and max points. While Vector2.Distance() could work, but it's quite computationally costly, defeating the purpose of the whole point.

In a case like this, the 'projection' property of the dot product can be utilized. I can take a dot product of 2 vectors. One points at the projected point from the min projected point, (the yellow dashed one, 'Projected'), and the other points from the min to the max projected point (the green one, 'Direction').

The length of the Direction could be computed beforehand, preferably in the constructor of a class or someplace similar.

Then, I can calculate the length of the yellow vector, denoted as l. Once I divide it once more by the length of direction, then I get the lerp factor.

// Prepare the projection matrix upon class initialization.
// (no need to re-initialize the very same matrix for each method invocation)
private readonly UnityEngine.Matrix4x4 mat = new(
    new Vector4(2, 0, 0, 1),
    new Vector4(0, 2, 0, 1),
    new Vector4(0, 0, 1, 0),
    new Vector4(0, 0, 0, 0));

private float ProjectionLerp1(Vector2 resolutionPoint)
{
    var projectedPointInHomgen = mat * resolutionPoint;
    var projectedPoint = projectedPointInHomgen / projectedPointInHomgen.w;

    // Calc the lerp factor
    var originAlignedPoint = (Vector2)projectedPoint - MinProjectionPoint;
    var lerpFactor = Vector2.Dot(originAlignedPoint, projLineDirection) / projLineLength;
    var normalizedLerpFactor = Mathf.Clamp01(lerpFactor / projLineLength); // normalize the factor to a range of (0, 1)
    return normalizedLerpFactor;
}

3.3. The Performance wasn't Good Enough

Not only there was no performance gain, but it turns out the performance has been severely degraded compared to the prior version.

I created a simple demo in Unity that accumulates the tick counts for each method. Every frame, it invokes each method once, counts its tick count, and accumulates it respectively.

Until frame number 4890, it took 31766 ticks in total to invoke the Atan version method, 42244 for the projection matrix version. (8 and 13 ticks at this particular frame)

There must be something happening under the hood. Let's take a look.

3.4. Examining the IL2CPP and Build Outcome

3.4.1 The Atan method post-il2cpp outcome

Plain and simple. It almost looks like a direct translation from the original C# code.

It calls the atan2f(), Mathf.Clamp(), and then subsequently calls the il2cpp_codegen_subtract and division.

Although It may seem like to make 4 function calls, only one of the calls to atan2f() happens. The rest of the calls get inlined by the C++ compiler. I could peer into the final outcome (GameAssembly.dll) through Ghidra.

(After all, atan2f() is the only function that gets invoked in the method)

3.4.2 The Projection Matrix method post-il2cpp outcome

Looks slightly more verbose than the atan version, but it doesn't seem so bad at a glance. The function calls, mostly vector/matrix arithmetic operations, look like they're going to easily get inlined by the compiler in the end.

Considering the atan2f() and the extra complexity it imposes with its nested if-else statements inside, this new version seems to have fewer instructions. Right?

Well, that wasn't the case. It has a larger instruction count, hence the degraded performance. The image below shows what it actually looks like after the C++ compiler.

As shown in the image, there's a bulky matrix initialization at the beginning of the decompiled function, followed by a function call to the Matrix4x4.Multiply(Vector3), which didn't get inlined.

Overall, this projection version shows no performance advantage over the atan2f() version inherently.

This bloated length of instructions will definitely NOT make this one faster than the previous one.

4. The vector way

Using a 4-by-4 matrix where a couple of simple vector arithmetic operations should do justice seems to be an overkill anyway. It could be worthwhile if it's a part of a series of affine transforms, but it isn't.

I could just get a t value and multiply it for each component of a resolution point.

private float ProjectionLerp2(Vector2 resolutionPoint)
{
    // Calc the point on the projection line
    var t = 2f / (resolutionPoint.x + resolutionPoint.y);
    var projectedPoint = t * resolutionPoint;

    // Calc the lerp factor: 
    // Project the projected point on the projection segment, to get the 't'
    // (dot product)
    var originAlignedPoint = projectedPoint - MinProjectionPoint;
    var lerpFactor = Vector2.Dot(originAlignedPoint, projLineDirection) / projLineLength;

    // Normalize the factor to a range of (0,1)
    var normalizedLerpFactor = Mathf.Clamp01(lerpFactor / projLineLength);

    return normalizedLerpFactor;
}

Disassembled	Decompiled

Compared to the previous matrix version, it looks much more lightweight. No single function call is involved. Let's count how many ticks it consumes.

Seems pretty performant this time.

5. Calc the x position only

Then I thought: why don't I just calculate the x-value only? I still can get the LerpFactor, and that's one less component to deal with. It could be more performant.

The projection line is just a plain, simple monotonous 1:1 mapping function (the green one). So just by designating one x-value, the corresponding y-value gets determined automatically, as the y-value is dependent on the x-value. (the other way around also makes sense, btw)

Simply performing the range comparison and finding the lerp factor of the yellow point on the x-axis in the green range will suffice.

The code would look like below:

private float ProjectionLerp3(Vector2 resolutionPoint)
{
    // Calc only the X of the projected point
    var t = 2f / (resolutionPoint.x + resolutionPoint.y);
    var projectedPointX = t * resolutionPoint.x;

    // Just Map the x value to the projection segment range.
    var clampedProjectedPointX = Mathf.Clamp(projectedPointX, MaxProjectionPoint.x, MinProjectionPoint.x);
    var lerpFactor = (MinProjectionPoint.x - clampedProjectedPointX) / (MinProjectionPoint.x - MaxProjectionPoint.x);

    return lerpFactor;
}

Disassembled	Decompiled

Well, the instruction count didn't shrink much this time. The previous vector version itself was quite efficient enough, taking advantage of the SIMD acceleration. Leaving not enough room to squeeze.

Overall Tick Counts

Over the course of 4890 frames:

• The atan2f() version has accumulated 31766 ticks for its invocations, with 8 ticks during this frame.
• The Projection Matrix version has 42244 ticks so far, with 13 ticks in this frame.
• The Vector version has 16428 ticks, with 5 ticks in this frame.
• The float version has 19761 ticks, with 4 ticks in this frame.

The projection matrix version couldn't bring much performance gain because of the matrix initialization overhead and the not-inlined function call.

The other 2 versions have successfully removed unnecessary operations, including function calls, resulting in fewer tick counts than the other previous 2 versions.

Conclusion

• When lerp-ing between 2 aspect ratios, you can't just lerp the number themselves.
• While lerp-ing, make sure your mid-point lies in the middle.
• Could take advantage of the homogeneous coordinate and a projection matrix, to make it evenly lerp-ed.

Performance-wise:

• Matrix-to-Vector multiplication doesn't necessarily always get inlined.
• Copy-constructing a matrix can take some time.
• Take a look at what your compiler emits. Examining the code sometimes isn't sufficient.