DEV Community: Chanwoo

Obtaining the Cubic Hermite Spline Derivative on an Arbitrary Interval

Chanwoo — Sun, 12 May 2024 04:50:05 +0000

Recently, I had to work with the Hermite spline for the first time in a while. I realized that I had forgotten some details about how it works. I decided to wrap it up here while I was at it.

1. Finding Hermite Spline

1.1. The 4 constraints

P1: The initial position, P
m1: The end point position, Q
P2: The slope/velocity at the initial position, P
m2: The slope/velocity at the end point, Q

(The red points represent P1, P2, while the blue slopes represent m1, and m2)

Since it is expressed as a cubic function that has 4 coefficients, it can have 4 constraints. The constraints of Hermite spline are the positions and the velocities (=slope) of the start and end points.

1.2. Finding the blending functions

Let's consider the Hermite spline to be obtained, denoted as P(t).

This, a nondescript cubic function, can be represented in a multiplication of 2 matrices.

A multiplication of a 1-by-4 matrix and a 4-by-1 matrix yields a 1-by-1 matrix. The one and only element in the matrix would be the evaluation of the spline given the t value.

To be able to evaluate the Hermite spline function, we need to figure out what the A matrix consist of. The A matrix comprises the 4 coefficients of a cubic polynomial function, the typical a, b, c, and d, as we all are familiar with.

The 4 constraints kick in here to determine the coefficients.

By definition, P1 represents the value P(t) where t = 0, and P2 does, P(1). m1 and m2 represents the slope of P(t) where t = 0, 1 respectively.

Substituting each t values in the T matrix with 0 and 1 accordingly, the constraints can be grouped as below.

To rearrange only the lateral sides of the equation using matrices, We could represent the matrix C (as in Constraints) as a multiplication of the coefficient matrix above and the A matrix.

By rearranging the equation around it, We could represent A in the form above.

Note: When finding an inverse matrix, https://matrix.reshish.com/inverse.php comes in quite handy.

By substituting the newly found A matrix into the existing expression P(t), we get the expression above.

We can take one step further and arrange the equation in the form of ax + by + cz + d = 0, but it's not gonna be useful. We will want to have it parameterized by the constraints, As your code will likely provide the constrains, not the coefficients. We will want to plug in them as-is.

The polynomial multiplied by each constraint is called a basis function or a blending function.

Then, in conclusion, the spline, and its derivative could be represented in the form below, where 0 <= t <= 1.

2. On an arbitrary interval `[x_{k}, x_{k+1}]`

The Hermite spline obtained above is valid only in the interval where t is in the canonical [0, 1] interval.

When t = 0, the evaluation and the slope would be P1, and m1 .
When t = 1, they would be P2 and m2.

In the ranges before and after 0 and 1, the evaluated values and the slopes will be completely meaningless to the given constraints setting, as it only works in the canonical [0, 1] range.

However, there are times when we want to move t along an arbitrary interval, but the canonical one. For example, what if we want to interpolate the values with the desired position and velocity when t is in the interval [3, 5]?

You might think: 'I can just normalize t to the length of the interval, to map [3, 5] to [0, 1]. Simple 1:1 interpolation. It would work, right?'

Nope, it didn't work.

For instance, let's say you want to find a Hermite spline with the constraints setting:

Starting point: (position: 1, velocity: 1)
- P1 = 1, m1 = 1
End point: (position: 0.7, velocity: 0)
- P2 = 0.7, m2 = 1

This will yield a spline valid only in the [0, 1] range. We could map the parameter t from it to the [3, 5] range by normalizing it as below.

t_{normalized} = (t - 3) / (5 - 3)

However, after the normalization, what you're getting is the curve in black color, whereas the actual curve we intended to have is in green color.

(The orange curve is the spline drawn in the normalized interval.)

When t is normalized as above, the original orange graph on the left ([0, 1] section) gets stretched left and right by the length of the target interval, then moved to the side by the beginning value of the target interval, yielding the black graph.

Stretching the graph left and right like this, the slope (=velocity) of the graph will be stretched as well, resulting in a stretched velocity different from the original. As this creates a spline that does not align with the intended constraints, this is an incorrect way to map the intervals.

What we want is the green graph, where the slope remains consistent across the range aligning with the initial constraints. To achieve this, we need to compensate for the reduced velocity that results from stretching the graph horizontally. To do this, simply multiply the velocity by the length of the interval to account for the stretch.

The spline curve P(x), the green one, on an arbitrary section below is as follows.

interval = [x_k, x_{k+1}]

p_k, p_{k+1} are the positions, m_k, m_{k+1} are the velocity.

You can see that the velocity constraints are multiplied by the length of the interval for stretch compensation.

This slope factor compensation is also mentioned in the Wikipedia article.

3. The Derivative of the Cubic Hermit Spline

The code I was working on required me to provide the derivative of the cubic Hermite spline.

The derivative of spline P(x) on an arbitrary interval is as follows.

(Since the constraints are mere scalars, they could be placed out of the differentiation brackets)

However, the blending functions are functions of t, not of x. Currently, they are differentiated with respect to x, so they need to be converted to differentiation with respect to t.

In fact, conveniently, the derivatives of each function with respect to x and t have the following relationship.

Let's take h_00(t) as an example.

First, substitute t with x and differentiate
Then, rearrange it for t,

as shown below.

The numerator of the final result is identical to the derivative of h00(t) with respect to t. Substituting yields the form on the right side above.

This pattern unfolds similarly for the other blending functions, leading to the following final result.

(As for the slope constraints, since (x_{k+1} - x_{k}) is already in the numerator, it cancels itself out.)

That is, in fact, the green graph shown above.

The black graph is the incorrectly stretched one.
The green graph is the correctly stretched, and compensated one.

Instead of code, I have parameterized the derivative of a cubic Hermite spline in the Desmos link below so that you can examine it.

Try changing the value of t_{slider} and observe how the slope graph touches and moves along the green curve.

https://www.desmos.com/calculator/h8taue3l9q

Note: When calculating derivatives, this website comes in handy: https://www.derivative-calculator.net/

Conclusion

Understanding the Hermite Spline Constraints:
- The cubic Hermite spline is defined by four constraints: the initial and final positions (P1 and P2) and their corresponding velocities (m1 and m2).
- Understanding these constraints and their relationship to the cubic polynomial's coefficients helps you draw the blending functions of the cubic Hermite spline.
- This step is crucial for mapping the spline's range and performing further differentiation.
Mapping from [0, 1] to another Arbitrary Range:
- Mapping the Hermite spline from the canonical [0, 1] interval to an arbitrary range [x, y] requires compensation for the interval's lateral scaling effect.
- The velocity constraints, need to be adjusted to maintain the initially desired spline across different ranges.
- This makes sure that the spline accurately interpolates the points and maintains the correct curvature.
Deriving the Spline's Derivative:
- It is necessary to find the derivative with respect to x and then represent it as a derivative with respect to t.
- Together with the arbitrary range mapping above, be aware that velocity compensation must also be included. Pay close attention to which terms are canceled out in this process.

Interpolating Between 2 Aspect Ratios in a Slightly Overcomplicated Way

Chanwoo — Mon, 06 May 2024 14:45:21 +0000

The other day, I had to work on a simple Lerp calculation for a Unity project. It was about lerping (= Linear Interpolation) an offset value from 3.0f to 4.0f, depending on the current window aspect ratio.

The initial specifications from the director were quite straightforward:

In 'Vertical Mode', when the game's window is narrow, the offset should be at the lower end (3.0f).
In 'Horizontal Mode', when the window is wide, the offset should be at the upper end (4.0f).
To prevent the offset from becoming too extreme, the value should be capped at each end:
- If the aspect ratio becomes smaller than 1080 * 1920 (a 9 : 16 aspect ratio), the offset should be capped at 3.0f, and should not be smaller than the cap.
- If the ratio exceeds 1920 * 1080 (a 16 : 9 ratio), the offset should be capped at 4.0f. No greater than that.
- When the aspect ratio is 1:1, the offset value should be at 50% of the range, i.e., (3.0 + 4.0) / 2 = 3.5.

By the way, users can freely adjust the window size to whatever they want, hence the need for capping.

The image above summarizes and visualizes the details:

Each capping end and the middle screen resolutions are denoted by the yellow boxes and the yellow 'resolution' points.
Since the game's window can be any size, any resolution points on the quadrant should be properly mapped to a lerp factor. (0.0f to 1.0f)
- If a resolution point lies along the y = x line, such as (800 * 800), the factor must be 0.5f.
- If a resolution point lies on or below the y = 1080 / 1920 x line, such as (480 * 270), the factor should be 0.0f.
- If it lies on or above the y = 1920 / 1080 x line, such as (100 * 1000), the factor should be 1.0f.

1. Can't just plug in the Aspect Ratio itself.

Simply plugging in the aspect ratio to obtain the lerpFactor wouldn't suffice. While it nicely interpolates between the 2 ends, it wouldn't yield the middle value when the aspect ratio is 1:1.

var minRatio = 1080/1920; // = 0.5625f
var maxRatio = 1920/1080; // = 1.7778f

// Capping the ratio
var aspectRatioClamped = Mathf.Clamp(aspectRatio, minRatio, maxRatio);

// Getting the lerpFactor (0 - 1)
var lerpFactor =  aspectRatioClamped - minRatio / (maxRatio - minRatio);

// NOTE: When the ratio is 1:1, the lerpFactor shall NOT be 0.5f.

When the ratio is 1:1, the lerp factor would be 1 - 0.5625 / (1.7778 - 0.5625) = 0.5372, slightly off from exactly 0.5f.

Converting the ratio into numerical form means setting the denominator to 1. In the graphical representation below, it's like obtaining the y-value of the intersection point between the line x = 1, and y = ratio * x.

For instance, if I am trying to get the lerp factor of a resolution of 960 * 640, the aspect ratio would be 960 / 640 = 1.5. To derive the lerp factor, I should first find the intersection point between x = 1, and y = 1.5 x, which is (1, 1.5). Then, I can perform the inverse-lerp with the y-value, to get a lerp factor of (1.5 - 0.5625) / (1.7778 - 0.5625) = 0.7714.
Any resolution proportional to this would have the same aspect ratio of 1.5, regardless of the size, thus yielding the same Lerp factor.

The x = 1 line vertically slices the ratio graph; as it's not perpendicular to each capping end, it naturally divides the range where slope < 1 and slope > 1 unevenly.

It's basically like performing an off-center projection (= oblique projection), so the center happens to be slightly off.

2. The `Atan()` way

To circumvent the unevenly divided range situation, one of the simplest approaches is to calculate the angles directly.

Get the angle between the resolution point to the x-axis (referred to as theta here).
Compare and perform an inverse-lerp operation on the angle within the range of (alpha to beta)
This ensures that the ranges around the midpoint are evenly divided; a resolution of 1:1 would yield a lerp factor of 0.5f.

FYI: (1920/1080 = 30 degrees, 1080/1920 = 60 degrees)

To represent this in C# code, it would appear as below:

private float AtanLerp(Vector2 resolutionPoint)
{
    var resolutionPointAngle = Mathf.Atan2(resolutionPoint.y, resolutionPoint.x);
    resolutionPointAngle = Mathf.Clamp(resolutionPointAngle, MinAngle, MaxAngle);

    float lerpFactor = (resolutionPointAngle - MinAngle) / (MaxAngle - MinAngle);
    return lerpFactor;
}

The code directly calculates the angle between the resolutionPoint to the x-axis, then checks whether the angle lies within the angular range of the 2 capped aspect ratios.

There's no need for projection to a vertical slice line; it works as intended. This was the approach I initially took in the codebase.

3. The Projection Matrix Way

After a couple of weeks from the implementation, I happened to glance over the graph I scribbled while rifling through the notebook. The graphs had somehow etched into my brain, and then I realized a function call to atan2() for each frame could be somewhat costly.

In reality, the performance cost of the function is virtually negligible. There are plenty of other parts that impose much heavier workloads. Nevertheless, I decided to delve into this during my free time for no reason.

The major issue with the 1st method (using the aspect ratio) is that the 'projection line' wasn't aligned with the projection range. As long as I set the 'projection line' (the green one) perpendicular to the mid-range slope (the dark-blue line: y = x), evenly dividing the projection range (the light-blue lines), there will be no more skewed projection issue.

Once I obtain the projected point (the yellow point where the green and yellow lines intersect) from a resolution point, I can then calculate the lerp factor with a few floating-point arithmetic operations by comparing the projected point with the capping points (the green points).

This approach sounded quite promising. The projection could be handled with a matrix-vector multiplication, which could potentially be SIMD accelerated. Additionally, a few floating-point additions, subtractions, and divisions wouldn't hurt the performance as much as atan2() would.

3.1. Projection Matrix

I can derive a projection matrix that transforms a resolution point into a 'projection point', utilizing the very same technique that every game engine uses, but in a way much simpler way.

A 'projected point' is simply the intersection point between the projection line and the line that connects the origin point and a resolution point.

The Latter, the line connecting the origin and the resolution point, could be represented as P(t) = t * P, where P is the resolution point, and t is a scalar parameter.

To get the intersection point (= projected point), plug in the parameterized P(t) to the projection line y = -x + 2, and then rearrange the equations to solve for t.

After rearranging the equation, it becomes clear how to determine the projected point from the resolution point P. For each element of P, multiply it by 2, then divide it by (y + x).

The matrix below will perform such a calculation with a homogeneous 2D vector (augmented with w = 1).

After homogeneous division, I can finally attain the projected point.

3.2. Calculating the Lerp Factor

Locating the projected point (x, y) is one thing, but I still need to derive the lerp factor from it.

To put it naively, I could compare the length between the projected point to a min or max projected point with the length between the min and max points. While Vector2.Distance() could work, but it's quite computationally costly, defeating the purpose of the whole point.

In a case like this, the 'projection' property of the dot product can be utilized. I can take a dot product of 2 vectors. One points at the projected point from the min projected point, (the yellow dashed one, 'Projected'), and the other points from the min to the max projected point (the green one, 'Direction').

The length of the Direction could be computed beforehand, preferably in the constructor of a class or someplace similar.

Then, I can calculate the length of the yellow vector, denoted as l. Once I divide it once more by the length of direction, then I get the lerp factor.

// Prepare the projection matrix upon class initialization.
// (no need to re-initialize the very same matrix for each method invocation)
private readonly UnityEngine.Matrix4x4 mat = new(
    new Vector4(2, 0, 0, 1),
    new Vector4(0, 2, 0, 1),
    new Vector4(0, 0, 1, 0),
    new Vector4(0, 0, 0, 0));

private float ProjectionLerp1(Vector2 resolutionPoint)
{
    var projectedPointInHomgen = mat * resolutionPoint;
    var projectedPoint = projectedPointInHomgen / projectedPointInHomgen.w;

    // Calc the lerp factor
    var originAlignedPoint = (Vector2)projectedPoint - MinProjectionPoint;
    var lerpFactor = Vector2.Dot(originAlignedPoint, projLineDirection) / projLineLength;
    var normalizedLerpFactor = Mathf.Clamp01(lerpFactor / projLineLength); // normalize the factor to a range of (0, 1)
    return normalizedLerpFactor;
}

3.3. The Performance wasn't Good Enough

Not only there was no performance gain, but it turns out the performance has been severely degraded compared to the prior version.

I created a simple demo in Unity that accumulates the tick counts for each method. Every frame, it invokes each method once, counts its tick count, and accumulates it respectively.

Until frame number 4890, it took 31766 ticks in total to invoke the Atan version method, 42244 for the projection matrix version. (8 and 13 ticks at this particular frame)

There must be something happening under the hood. Let's take a look.

3.4. Examining the IL2CPP and Build Outcome

3.4.1 The Atan method post-il2cpp outcome

Plain and simple. It almost looks like a direct translation from the original C# code.

It calls the atan2f(), Mathf.Clamp(), and then subsequently calls the il2cpp_codegen_subtract and division.

Although It may seem like to make 4 function calls, only one of the calls to atan2f() happens. The rest of the calls get inlined by the C++ compiler. I could peer into the final outcome (GameAssembly.dll) through Ghidra.

(After all, atan2f() is the only function that gets invoked in the method)

3.4.2 The Projection Matrix method post-il2cpp outcome

Looks slightly more verbose than the atan version, but it doesn't seem so bad at a glance. The function calls, mostly vector/matrix arithmetic operations, look like they're going to easily get inlined by the compiler in the end.

Considering the atan2f() and the extra complexity it imposes with its nested if-else statements inside, this new version seems to have fewer instructions. Right?

Well, that wasn't the case. It has a larger instruction count, hence the degraded performance. The image below shows what it actually looks like after the C++ compiler.

As shown in the image, there's a bulky matrix initialization at the beginning of the decompiled function, followed by a function call to the Matrix4x4.Multiply(Vector3), which didn't get inlined.

Overall, this projection version shows no performance advantage over the atan2f() version inherently.

This bloated length of instructions will definitely NOT make this one faster than the previous one.

4. The vector way

Using a 4-by-4 matrix where a couple of simple vector arithmetic operations should do justice seems to be an overkill anyway. It could be worthwhile if it's a part of a series of affine transforms, but it isn't.

I could just get a t value and multiply it for each component of a resolution point.

private float ProjectionLerp2(Vector2 resolutionPoint)
{
    // Calc the point on the projection line
    var t = 2f / (resolutionPoint.x + resolutionPoint.y);
    var projectedPoint = t * resolutionPoint;

    // Calc the lerp factor: 
    // Project the projected point on the projection segment, to get the 't'
    // (dot product)
    var originAlignedPoint = projectedPoint - MinProjectionPoint;
    var lerpFactor = Vector2.Dot(originAlignedPoint, projLineDirection) / projLineLength;

    // Normalize the factor to a range of (0,1)
    var normalizedLerpFactor = Mathf.Clamp01(lerpFactor / projLineLength);

    return normalizedLerpFactor;
}

Disassembled	Decompiled

Compared to the previous matrix version, it looks much more lightweight. No single function call is involved. Let's count how many ticks it consumes.

Seems pretty performant this time.

5. Calc the x position only

Then I thought: why don't I just calculate the x-value only? I still can get the LerpFactor, and that's one less component to deal with. It could be more performant.

The projection line is just a plain, simple monotonous 1:1 mapping function (the green one). So just by designating one x-value, the corresponding y-value gets determined automatically, as the y-value is dependent on the x-value. (the other way around also makes sense, btw)

Simply performing the range comparison and finding the lerp factor of the yellow point on the x-axis in the green range will suffice.

The code would look like below:

private float ProjectionLerp3(Vector2 resolutionPoint)
{
    // Calc only the X of the projected point
    var t = 2f / (resolutionPoint.x + resolutionPoint.y);
    var projectedPointX = t * resolutionPoint.x;

    // Just Map the x value to the projection segment range.
    var clampedProjectedPointX = Mathf.Clamp(projectedPointX, MaxProjectionPoint.x, MinProjectionPoint.x);
    var lerpFactor = (MinProjectionPoint.x - clampedProjectedPointX) / (MinProjectionPoint.x - MaxProjectionPoint.x);

    return lerpFactor;
}

Disassembled	Decompiled

Well, the instruction count didn't shrink much this time. The previous vector version itself was quite efficient enough, taking advantage of the SIMD acceleration. Leaving not enough room to squeeze.

Overall Tick Counts

Over the course of 4890 frames:

The atan2f() version has accumulated 31766 ticks for its invocations, with 8 ticks during this frame.
The Projection Matrix version has 42244 ticks so far, with 13 ticks in this frame.
The Vector version has 16428 ticks, with 5 ticks in this frame.
The float version has 19761 ticks, with 4 ticks in this frame.

The projection matrix version couldn't bring much performance gain because of the matrix initialization overhead and the not-inlined function call.

The other 2 versions have successfully removed unnecessary operations, including function calls, resulting in fewer tick counts than the other previous 2 versions.

Conclusion

When lerp-ing between 2 aspect ratios, you can't just lerp the number themselves.
While lerp-ing, make sure your mid-point lies in the middle.
Could take advantage of the homogeneous coordinate and a projection matrix, to make it evenly lerp-ed.

Performance-wise:

Matrix-to-Vector multiplication doesn't necessarily always get inlined.
Copy-constructing a matrix can take some time.
Take a look at what your compiler emits. Examining the code sometimes isn't sufficient.

Deploying Unity Executable as a Windows Service

Chanwoo — Wed, 01 May 2024 17:15:28 +0000

When you're running out of rendering budget in your game project, offloading some less time-critical rendering operations to another machine can be an option. Recently at work, we created an image server using Unity as a method to offload some of the rendering workload from the game client program and added it to the server deployment script as part of our server group.

However, deploying a Unity-built executable that utilizes the graphics API as a Windows service didn't go as smoothly as I had expected initially. In this post, I am going to walk through some of the technical difficulties I faced while deploying the image server.

The Image Server

The image server is responsible for accepting HTTP requests from clients and, returns the rendered image in the response, upon completing the rendering processes as specified in the request.

In practice, the server will render some elements that the client doesn’t immediately need but will within a few seconds. This could be the map image of a procedurally generated dungeon or an image of a randomized NPC appearance, possibly displayed in a corner of the UI. (By the way, neither of these examples was our case.) Essentially, it handles whatever tasks the client can delay for a short time before retrieving the actual image. I won't delve much deeper into this aspect, as it heavily depends on your project's requirements and what you choose to render on the server.

Setting aside the rendering processes, one of the simplest forms to illustrate the server's behavior might look like the code below. It's essentially a basic HTTP server that listens on port 3000 and returns hard-coded HTML in response.

using System.Net;
using System.Threading;
using UnityEngine;

public class MyHttpServer : MonoBehaviour
{
    private HttpListener listener;
    private Thread listenerThread;

    void Start()
    {
        listener = new HttpListener();
        listener.Prefixes.Add("http://localhost:3000/");
        listener.Start();
        Debug.Log("Listening...");

        listenerThread = new Thread(new ThreadStart(Listen));
        listenerThread.Start();
    }

    void Listen()
    {
        while (true)
        {
            HttpListenerContext context = listener.GetContext();
            HttpListenerRequest request = context.Request;
            HttpListenerResponse response = context.Response;

            string responseString = "<html><body>Hello World!</body></html>";
            byte[] buffer = System.Text.Encoding.UTF8.GetBytes(responseString);

            response.ContentLength64 = buffer.Length;
            System.IO.Stream output = response.OutputStream;
            output.Write(buffer, 0, buffer.Length);
            output.Close();
        }
    }

    private void OnApplicationQuit()
    {
        listener.Stop();
        listenerThread.Abort();
    }
}

Attach it to an arbitrary GameObject in the scene, then build the executable. Once it's up and running, you should be able to see the simple html code mentioned above returned as a response in your web browser at the address (localhost:3000, in this case).

(Let's pretend it's returning an image instead of the HTML)

Issues with the Windows service

The servers were deployed as services on Windows machines, primarily to simplify the automatic starting and stopping of server processes when the machine is turned on and off.

The issues kick in right at this point. A Windows service process runs with admin privileges in a special session called 'Session 0', while other standard processes run in each user's interactive logon session.

In Session 0, a service process is meant to run as a 'background' process. A background process in Windows does not inherit the capability to use the GUI system; it is unable to create a window or use graphic APIs, such as DirectX or OpenGL. Even if you spawn a service process from a Unity-built executable, it is initiated as a background process. Being unable to use the GUI, it gets stuck at the initialization stage, even before the HTTP server MonoBehavior is attached to the GameObject.

This means that the image server won't function when it runs as a regular service running on Session 0. Since its primary purpose is to 'render images', which requires full utilization of the GUI and graphics API, it should operate in an interactive logon session to enable this functionality. Not as a regular background service process.

What if I run it as a service anyway?

The process will be spawned, but it will get stuck in the graphics API initialization stage.

(The image server process was created as a 'background' process, stuck in the init stage)

In the image above, the HttpServerTest process is up and running. However, since it couldn't complete the initialization process, it was unable to bind to port 3000. The output of the command netstat -aon | findstr :3000 reveals that no process is bound to the designated port number. Consequently, the command Invoke-WebRequest, which sends a GET http request, fails to connect to the given URL.

If you create a dump of the stuck process, you can locate where it's actually stuck. After opening the dump and examining the call stack, I noticed it was stuck at the resolution setting. The exact message the program is trying to display, but couldn't because it couldn't use GUI, becomes apparent.

Use PsExec to run it on a user session

A sysinternals program PsExec can be quite helpful in this case. It allows you to execute other programs on another user's session, even on a remote computer, if you have provided the right credentials.

> psexec \\RemoteMachine -u username -p password -d -i <session id> <program to execute>

With the command line option -i, you can specify the session id the 'program to execute' will be running on. A proper session ID can easily be obtained from the command, query session.

(You can use PsExec to execute a program on either an 'Active' or a 'Disconnected' session.)

With the commands mentioned above, you can write a simple program or a PowerShell script that utilizes PsExec to execute your server in a user session. Please be advised that you need to log in to the target machine at least once, to create an interactable user logon session.

But PsExec requires some extra configs

The commands should work. as they did for me in the proof-of-concept script. However, I soon realized that PsExec requires additional configurations on the recipient machines. I was concerned that the security department, responsible for auditing server configurations and deploying live servers, might not approve these changes due to the significant security risks this might impose. In my case especially, this meant that I needed to find another way to circumvent the direct usage of PsExec.

The additional configurations include:

Opening up the TCP 445 port, by adding allowing rules to the firewall.
Turning off the UAC (User Account Control).
Enabling the 'File and Printer Sharing' option.

So instead of executing PsExec from the deployment script directly, I chose to bundle the PsExec executable with the image server, let the recipient machine run the PsExec locally on the machine, via the Invoke-Command executed remotely.

# `query session` command
$rawOutput = Invoke-Command -ComputerName $remoteIp -Credential $credential -ScriptBlock {
    param ($remoteUsername)
    return query session $remoteUsername
} -ArgumentList $remoteUsername

# parse the output and obtain a proper session id
$sessionId = ... # could be 1, or 2, etc.

Invoke-Command -ComputerName $remoteIp -Credential $credential -ScriptBlock {
    param ($sessionId)
    & 'C:\Path\To\PsExec' /accepteula -i $sessionId -d 'C:\Path\To\ImageServer.exe' 2>&1
} -ArgumentList $sessionId

I configured the deployment script in the same way as above. Upon successfully executing it, I could see the image server was up and running, with its GUI functionality initialized properly. It successfully bound the process to port number 3000, being able to accept http requests and respond with the expected payload.

(In this case, unlike above, the process is created as a 'foreground' process)

wrote another PoC program in C# that does the same thing as the Powershell script above and made it run as a service via NSSM, worked like a charm.

Conclusion

There are several key considerations when running a Unity-built application as a Windows service:

1. Windows services run as background processes

Windows services are primarily designed to run non-interactive processes in the background. Consequently, any process run under a Windows service is executed as a background process.

2. To make use of Graphics API, it should be a foreground process

If you run Unity output as a background process, it won't be able to get past the GUI initialization phase. Naturally, all the Unity functionalities will not be able to work, including network initialization.

3. Execute the process via `PsExec`

PsExec will allow you to specify in which session your program will run. It's possible to write a server deployment script or a program that creates a logon session and executes the server via PsExec on the session.
With the help of a program like NSSM, it'd be easy to make it into a Windows service.

DEV Community: Chanwoo

Obtaining the Cubic Hermite Spline Derivative on an Arbitrary Interval

1. Finding Hermite Spline

1.1. The 4 constraints

1.2. Finding the blending functions

2. On an arbitrary interval [x_{k}, x_{k+1}]

3. The Derivative of the Cubic Hermit Spline

Conclusion

Interpolating Between 2 Aspect Ratios in a Slightly Overcomplicated Way

1. Can't just plug in the Aspect Ratio itself.

2. The Atan() way

3. The Projection Matrix Way

3.1. Projection Matrix

3.2. Calculating the Lerp Factor

3.3. The Performance wasn't Good Enough

3.4. Examining the IL2CPP and Build Outcome

3.4.1 The Atan method post-il2cpp outcome

3.4.2 The Projection Matrix method post-il2cpp outcome

4. The vector way

5. Calc the x position only

Overall Tick Counts

Conclusion

Deploying Unity Executable as a Windows Service

The Image Server

Issues with the Windows service

What if I run it as a service anyway?

Use PsExec to run it on a user session

But PsExec requires some extra configs

Conclusion

1. Windows services run as background processes

2. To make use of Graphics API, it should be a foreground process

3. Execute the process via PsExec

2. On an arbitrary interval `[x_{k}, x_{k+1}]`

2. The `Atan()` way

3. Execute the process via `PsExec`