[Unity] Encoding Data into Pixels (Part 1): CG Shaders

For a project, I want to implement a feature which allows the picking of gameobjects by color. I was first introduced to this technique here.

I want pixel perfect object selection, and this approach is relatively straightforward. All it involves is painting each object in a distinct color, this color being a unique ID associated to each object. Then, we can simply see what color, and thus what object the player is pointing at.

Ideation

We create a new Universal Renderer Data since we want to color each gameobject in a different color, but don't want this effect to be visible on the main camera. We want this effect to be exclusive to a secondary camera which we will use only for the color picking action.

Setup

1. Locate the Universal Render Pipeline Asset used in your project. (To identify this one you can open Project Settings and then navigate to the Graphics section.)
2. Add a new field to its Renderer List under the Rendering section.
3. Create a new Universal Renderer Data asset. (Create -> Rendering -> URP Universal Renderer)
4. See its details in the Inspector, and then click on Add Render Feature, selecting the option Render Objects (Experimental). This one in particular allows us to apply the same material to every gameobject in our scene.
5. Inside the section for the newly created Render Feature, under the Filters section, select the Everything option for the Layer Mask field. In this case, I will want to paint every single GameObject.
6. Inside the section for the newly created Render Feature, click on Overrides. A Material field should be visible.
7. Create a new Material.
8. Create a new Shader, we will use this one later to color each gameobject in a unique color.
9. Assign the Shader to the newly created Material.
10. Assign the newly created Material to the Material field in the Render Feature section.
11. Assign the Universal Renderer Data to the previously created Renderer List field, created in step 2.
12. Inspect your secondary camera.
13. Select the newly created Renderer from the Renderer field in the Rendering section.

Hello World Shader

If you have followed along the steps above, recreating the Renderer, I have added this small section that tests if you did the setup correctly.

To test the correctness of the setup, you can use the following Shader code.

Shader "Custom/UniqueColoring" {
    SubShader {
        Tags { "RenderType"="Opaque" }
        LOD 100

        Pass {
            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            #include "UnityCG.cginc"

            struct appdata {
                float4 vertex : POSITION;
            };

            struct v2f {
                float4 vertex : SV_POSITION;
            };

            v2f vert (appdata v) {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                return o;
            }

            fixed4 frag (v2f i) : SV_Target {
                // always return red color
                return fixed4(1.0, 0.0, 0.0, 1.0);
            }
            ENDCG
        }
    }
}

If everything was setup correctly, any GameObject you see through the second camera should now be red.

Bits & Bits

As of right now, our Shader only returns the color red for each fragment, which is determined by the following line:

return fixed4(1.0, 0.0, 0.0, 1.0);

We want to store a number in at least one of the color channels, say the red one for example.

The fragment Shader can be defined to return either a fixed4, half4, or a float4 data structure, which represent respectively, 4 11-bit numbers, 4 16-bit numbers, and 4 32-bit numbers. [Source]

To maximize the amount of data we can encode in the color, will choose to go with our fragment Shader returning a float4 data structure.

32-bits

The following section is very important to make sure everything works as expected.

We need to pay attention that when the fragment Shader returns 4 32-bit values, that any other functions these are passed to, do not make us lose any precision by converting the values to a data structure of lower precision.

Specifically, for reading colors I will be using my approach from my last post.

The following outlines the entire pass from the Shader to the final reading:

1. The Shader returns 4 32-bit values.
2. The image of my second camera is rendered into my Render Texture.
3. My Compute Shader reads a pixel from the Render Texture, and stores it in a buffer.
4. A script I have reads the value from the buffer.

Something that I missed early on when developing this solution is the precision value of the color channels for the Render Texture, which by default, are set to a very low precision value.

Best Render Texture

A Render Texture object has many properties for its RenderTextureFormat enumeration.

Here's a screenshot from the official documentation including some of them:

In our case we select the last option visible in the screenshot above: "ARGBFloat", making each color channel 32-bits, such that we preserve the precision level from the Shader.

In my case, I am not creating the Render Texture programmatically. I select the R32G32B32A32_SFLOAT option under the Color Format field.

In any case, we need to make sure that there is no loss in precision, and that at each step we preserve the 4 32-bit values without making any change to them.

In line with another great article:
Additional Considerations:

1. We use point filtering, since we don't want anti-aliasing or filtering.
2. We don't need MipMaps.
3. The Skybox should just be a solid color, filled with 0s as can be seen below:

How much is 32-bits?

In a 32-bit system, each bit can represent two possible values: 0 or 1. Since there are 32 bits in total, we can calculate the number of possible values by raising 2 to the power of 32:

2^32 = 4,294,967,296

Therefore, there are 4,294,967,296 different values that can be represented in 32 bits.

Conclusion

Everything should work now.
Try changing the Shader's color to have something like 0.5 or 0.001 in its red color channel, to then see if this is indeed the value output in the console (assuming you are logging every reading).