[Unity] Fast Pixel Reading (Part 1): Render Textures & Compute Shaders

Intro

For a project, I needed to call ReadPixels every single frame, i.e. in the Update function.

As it turns out, this is a very costly operation, because it involves communicating information from the GPU to the CPU.

The following post details how I improved the speed at which I get information from the GPU to the CPU from 0.257ms to 0.075ms, an improvement of 70%.

Foreword

For this post I replicate my findings from a Unity project I built from scratch so that it can be replicated easily if wanted.

The following are the steps I took, starting in Unity Hub.
I also include simple performance evaluations at every step.

With "baseline" I shall refer to the fps count level measured in my "3D (URD)" Unity scene, before having done anything to it after creating it.

Testing

For evaluating performance I use the following script, which logs the average fps count over the first 10 seconds of running the game.

using UnityEngine;

public class AverageFPSCounter : MonoBehaviour
{
    private float startTime;
    private int frameCount;

    private void Start()
    {
        startTime = Time.time;
        frameCount = 0;
    }

    private void Update()
    {
        frameCount++;

        if (Time.time - startTime >= 10f)
        {
            float averageFPS = frameCount / (Time.time - startTime);
            Debug.Log("Average FPS over 10 seconds: " + averageFPS);

            // Optionally, you can stop the script from updating further.
            enabled = false;
        }
    }
}

Setup

1. Create a new Unity project using the "3D (URP)" template. (I'm using Unity version 2021.3.16f1)
2. Create a new "Camera" game object.
3. Remove the "Audio Listener" component.

Running the game now with 2 active cameras in the scene naturally tanks performance by quite a bit.

Performance Evaluation: Baseline drop of 43%

RenderTexture

In my case I just want to read the color of pixels from a separate camera, which does not need to be rendered to the player, which is why I just render it into a RenderTexture, wich is a lot more efficient.

1. Create a RenderTexture.
2. Assign it to the "Output Texture" of the secondary camera.

Performance Evaluation: Baseline drop of 11%

ReadPixel

I created a simple script that reads the color of the first pixel from the RenderTexture, and does this every single frame.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class PixelReader : MonoBehaviour
{
    [SerializeField] private RenderTexture _renderTexture;
    private Texture2D texture2D;

    private void Start()
    {
        texture2D = new Texture2D(_renderTexture.width, _renderTexture.height, TextureFormat.RGBA32, false);
    }

    private void Update()
    {
        ReadFirstPixel();
    }

    private void ReadFirstPixel()
    {
        // Copy the RenderTexture's pixels to the Texture2D
        RenderTexture.active = _renderTexture;
        texture2D.ReadPixels(new Rect(0, 0, _renderTexture.width, _renderTexture.height), 0, 0);
        texture2D.Apply();

        // Get the first pixel's color
        Color firstPixelColor = texture2D.GetPixel(0, 0);
    }
}

1. Attach the script to the secondary camera.
2. Drag previously created RenderTexture into the visible "Render Texture" field.

Performance evaluation: Baseline drop of 23%

Line-by-Line Code Performance Evaluation

Executing the ReadFirstPixel function 100 times, the average runtime for it is 0.257ms.

I also made separate evaluations for all lines within the ReadFirstPixel function. The number in brackets next to the line number indicates the average runtime for the line across 100 runs.

Line 1 (0.011ms):

RenderTexture.active = _renderTexture;

Line 2 (0.213ms):

texture2D.ReadPixels(new Rect(0, 0, _renderTexture.width, _renderTexture.height), 0, 0);

Line 3 (0.064ms):

texture2D.Apply();

Line 4 (0.007ms):

Color firstPixelColor = texture2D.GetPixel(0, 0);

Conclusion: As expected, ReadPixels is indeed the function that takes the longest to execute. Let's try to optimize this!

Compute Shader

Instead of using ReadPixels, we will be creating a Compute Shader to transfer the color data from the GPU to the CPU in a faster way.

The following is my Compute Shader.

#pragma kernel ReadFirstPixel

// Input texture
Texture2D<float4> inputTexture;

// Output buffer
#pragma bind_buffer(name: outputBuffer, binding: 0)

RWStructuredBuffer<float4> outputBuffer;

[numthreads(1, 1, 1)]
void ReadFirstPixel(uint3 id : SV_DispatchThreadID)
{
    // Read the color of the first pixel
    float4 color = inputTexture.Load(int3(0, 0, 0));

    // Store the color in the output buffer
    outputBuffer[0] = color;
}

The following is my new version of the PixelReader script, which now uses the compute shader.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System;

public class PixelReader : MonoBehaviour
{
    [SerializeField] private RenderTexture inputTexture;
    private Texture2D texture2D;

    [SerializeField] private ComputeShader ReadFirstPixel;
    private ComputeBuffer outputBuffer;
    private int kernelID;

    private void Start()
    {
        texture2D = new Texture2D(inputTexture.width, inputTexture.height, TextureFormat.RGBA32, false);

        // Create the output buffer
        outputBuffer = new ComputeBuffer(1, sizeof(float) * 4);

        // Get the kernel ID of the ReadFirstPixel function
        kernelID = ReadFirstPixel.FindKernel("ReadFirstPixel");

        // Set the input texture and output buffer to the shader
        ReadFirstPixel.SetTexture(kernelID, "inputTexture", inputTexture);
        ReadFirstPixel.SetBuffer(kernelID, "outputBuffer", outputBuffer);
    }

    private void Update()
    {
        FastReadPixel();
    }

    private void FastReadPixel()
    {
        // Dispatch the compute shader
        ReadFirstPixel.Dispatch(kernelID, 1, 1, 1);

        // Read the result from the output buffer
        float[] outputArray = new float[4];
        outputBuffer.GetData(outputArray);

        // Extract the color components from the output array
        Color color = new Color(outputArray[0], outputArray[1], outputArray[2], outputArray[3]);
    }

    void OnDestroy()
    {
        // Release the output buffer
        outputBuffer.Release();
    }
}

Performance evaluation: Baseline drop of 25%

Line-by-Line Code Performance Evaluation

Executing the new FastReadPixel function 100 times, the average runtime for it is 0.075ms.

I also made separate evaluations for all lines within the FastReadPixel function. The number in brackets next to the line number indicates the average runtime for the line across 100 runs.

Line 1 (0.005ms):

ReadFirstPixel.Dispatch(kernelID, 1, 1, 1);

Line 2 (0.001ms):

float[] outputArray = new float[4];

Line 3 (0.073ms):

outputBuffer.GetData(outputArray);

Line 4 (0.001ms):

Color color = new Color(outputArray[0], outputArray[1], outputArray[2], outputArray[3]);