DEV Community: kurohuku

Get controller device index without GetTrackedDeviceIndexForControllerRole()

kurohuku — Thu, 10 Jul 2025 11:11:53 +0000

Overview

GetTrackedDeviceIndexForControllerRole() and GetControllerRoleForTrackedDeviceIndex() are deprecated.
This guide is for those who want to get the controller's Device Index using another method.
Get the controller's Device Index from a Pose action by IVR Input (SteamVR Input).

Deprecated Functions

In a previously SteamVR Overlay tutorial, , I used GetTrackedDeviceIndexForControllerRole() to get the controller's Device Index like this:

var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);

However, GetTrackedDeviceIndexForControllerRole() is marked as deprecated in the header file's comment. So, this I try to get the Device Index using IVRInput (SteamVR Input).
Although it's deprecated, it's still used within OpenVR's own code, so personally, I don't see any immediate issues with continuing to use it.

Reference:
https://github.com/ValveSoftware/openvr/blob/master/headers/openvr.h

Create Unity Project

Create a new project. I'm using the following environment:

Unity 6.1
URP Template

Delete all objects from the scene to make it empty.

Instlal SteamVR Plugin

Install the SteamVR Plugin via the Package Manager.

https://assetstore.unity.com/packages/tools/integration/steamvr-plugin-32647

Controller Binding

Follow these steps to bind controller Poses to actions:

Window -> SteamVR Input
Select NO fro "Copy Examples"
For "ActionSet Name", enter "DeviceIndex" (any name is fine). Select "per hand" mode.
Create two Actions: LeftHandPose and RightHandPose.
Set the Type of both to "pose".

Click "Save and generate".
Click "Open binding UI".
Click "Create New Binding".

Click the "Poses" button.

Assign the LeftHandPose and RightHandPose actions to Left Hand Raw and Right Hand Raw respectively.

Click "Replace Default Binding" to save it as the default.

If the "Poses" button is not shown

Regarding the "Poses" button mentioned above, I've not investigated details, but it sometimes doesn't appear.
It should appear if a pose action is defined in actions.json, but I've encountered cases where it didn't show up after several attempts.

If it's not showing, try restarting SteamVR, creating the actions.json file directly, or bringing in actions.json from another app. You might need to try various things.

Script Creation

Create a script named GetControllerDeviceIndex.cs.
Here, we'll get the Device Index for the left-hand controller.

For details on how to get actions and other code specifics, please refer to this tutorial.
The important part starts from where controllerOriginInfo is obtained further down in Update().

using System;
using Valve.VR;
using UnityEngine;

class GetControllerDeviceIndex : MonoBehaviour
{
    ulong actionSetHandle = OpenVR.k_ulInvalidActionSetHandle;
    ulong actionHandle = OpenVR.k_ulInvalidActionHandle;

    void Start()
    {
        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            Debug.LogError("Failed to set action manifest path: " + error);
            return;
        }

        error = OpenVR.Input.GetActionSetHandle("/actions/DeviceIndex", ref actionSetHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get action set /actions/DeviceIndex: " + error);
        }

        error = OpenVR.Input.GetActionHandle("/actions/DeviceIndex/in/LeftHandPose", ref actionHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get action /actions/DeviceIndex/in/LeftHandPose: " + error);
        }
    }

    void Update()
    {
        var actionSetList = new VRActiveActionSet_t[]
        {
            new VRActiveActionSet_t()
            {
                ulActionSet =  actionSetHandle,
                ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
            }
        };

        var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
        var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to update action state: " + error);
        }

        var result = new InputPoseActionData_t();
        var poseActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputPoseActionData_t));
        error = OpenVR.Input.GetPoseActionDataForNextFrame(actionHandle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref result, poseActionSize, OpenVR.k_ulInvalidInputValueHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get pose action data: " + error);
        }

        // Passing the action's activeOrigin to GetOriginTrackedDeviceInfo() to get the device that originated the action.
        var controllerOriginInfo = new InputOriginInfo_t();
        var inputOriginInfoSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputOriginInfo_t));
        error = OpenVR.Input.GetOriginTrackedDeviceInfo(result.activeOrigin, ref controllerOriginInfo, inputOriginInfoSize);
        if (error != EVRInputError.None)
        {
            Debug.LogError("Failed to get left controller origin info: " + error);
            return;
        }

        // Get the Device Index of the obtained device.
        var deviceIndex = controllerOriginInfo.trackedDeviceIndex;
        Debug.Log($"Left Controller Device Index: {deviceIndex}");
    }
}

Execution

Create an empty Game Object in the scene, add the above component to it, and run the scene.
The Device Index of the left-hand controller will be displayed in the console.

After obtaining the Device Index from the action in this manner, you can use it by passing it as an argument to SetOverlayTransformRelative(), for example.

SteamVR Overlay with Unity: Appendix

kurohuku — Tue, 18 Jun 2024 15:11:43 +0000

This part contains where to look for more detailed information.

OpenVR repository

OpenVR’s official repository is the most important

https://github.com/ValveSoftware/openvr

I recommend reading the wiki at first.

https://github.com/ValveSoftware/openvr/wiki/API-Documentation

Detailed information that is not on the wiki tends to be written in C++ header file comments. If you are looking for some feature, try to find the header file.

https://github.com/ValveSoftware/openvr/blob/master/headers/openvr.h

Anything else, try to find from the issue.

https://github.com/ValveSoftware/openvr/issues

SteamVR Unity Plugin repository

SteamVR Unity Plugin repository has utilities for Unity.

https://github.com/ValveSoftware/steamvr_unity_plugin

Here is the document.

https://valvesoftware.github.io/steamvr_unity_plugin/api/index.html

Other repositories

Open source projects like OVR Advanced Settings are helpful.

https://github.com/OpenVR-Advanced-Settings/OpenVR-AdvancedSettings

Show 3d objects with overlay

Overlay can show 3d objects with stereo image.
For example, the VR paint app Vermillion has overlay mode that displays 3d objects.

I don’t have enough information but list up methods I know.
(The below information may contain incorrect information.)

Method 1: SideBySide

Pass VROverlayFlags.SideBySide_Parallel to SetOverlayFlags(). This enables stereo images for overlay.
Side by side image is an image that the left half to the left eye and the right half to the right eye then it makes 3d.
Create two cameras for each eye in Unity, and draw the camera output horizontally onto the texture to make side by side texture.

Method 2: Stereo Panorama

Pass VROverlayFlags.StereoPanorama to SetOverlayFlags(). This enables stereo panorama for overlay. Like the side by side, stereo panorama uses two areas of the image for each eye.

Method 3: Overlay projection

SetOverlayTransformProjection() shows overlay for the specific eye only.
You can get the eye position by transforming Origin -> HMD -> Eye with GetEyeToHeadTransform().

// Detect dark theme var iframe = document.getElementById('tweet-1566307113697423360-809'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1566307113697423360&theme=dark" }

Method 4: Combine planes

If the desired 3d object is a simple form like a cube, you can make 3d objects with 2d planes. For example, create a cube with 6 overlays or a cylinder with a curved overlay with SetOverlayCurvature().

I use this method in OVR Locomotion Effect. It makes a large grid cube that encloses the player with 6 overlays. Also, the wind effect is made with a curved overlay as a tube and encloses the player’s HMD to create a sense of depth.

Find with keyword stereo or sidebyside in openvr.h for 3d objects.

Get the HMD output

GetMirrorTextureD3D11() or GetMirrorTextureGL() of OpenVR.Compositor read the HMD output.

Get controller device index without deprecated function `GetTrackedDeviceIndexForControllerRole`

Here is an article how to get a controller device index with IVR Input (SteamVR Input).

SteamVR Overlay with Unity: Controller Input

kurohuku — Tue, 18 Jun 2024 15:01:34 +0000

In this part, we will make the watch hidden by default and show it in a few seconds by pressing a controller button.

There are various ways to get controller input with Unity. This time, we will use OpenVR Input API (SteamVR Input).

Prerequisite setting

Open the SteamVR Settings.

Toggle Advanced Settings.

Toggle Developer > Enable debugging options in the input binding user interface to On.

Create action manifest

SteamVR Input uses abstracted actions instead of direct access to the physical buttons.
The developer predefines actions and may create default mappings for each controller. Users can assign physical buttons to the actions in the SteamVR Input setting window.

First, create the action manifest that defines actions in JSON format. We use the Unity SteamVR Plugin GUI tool to generate the action manifest easily.

Generate action manifest

Select Unity menu Window > SteamVR Input.

It asks you whether you want to use a sample action manifest file.
This time we will make up from the first so select No.

Input the action sets name to “Watch”, and select the below dropdown to per hand. An application can have multiple action sets.

Per hand means it can map different actions to the buttons of the left and right-hand controller separately.
Mirrored means it can map actions to only one controller and the other buttons are mirrored to the mapping.

Click the NewAction name that is in the In box of the Actions section.
The “Action Details” is shown to the right. Change the Name to “WakeUp”.
Click the “Save and generate” button at the left bottom to generate an action manifest file.

It will generate a new action manifest file as StreamingAssets/SteamVR/actions.json.

{
  "actions": [
    {
      "name": "/actions/Watch/in/WakeUp",
      "type": "boolean"
    }
  ],
  "action_sets": [
    {
      "name": "/actions/Watch",
      "usage": "leftright"
    }
  ],
  "default_bindings": [],
  "localization": []
}

Create default binding

Click the right bottom Open binding UI button.

If any VR controller is not shown, check your HMD is recognized from SteamVR.

Click the “Create New Binding”.

Set the Y button to show the watch for a few seconds. If your controller doesn’t have the Y button, use another button instead.

Click the + button to the right of the Y Button.

Select BUTTON.

Select None to the right of the Click, then select wakeup action.

Click the left bottom check icon to save changes.

Click the right bottom Replace Default Binding.

Click Save.

At this point, a new default binding file is added to the action manifest.

actions.json

{
   "action_sets" : [
      {
         "name" : "/actions/Watch",
         "usage" : "single"
      }
   ],
   "actions" : [
      {
         "name" : "/actions/Watch/in/WakeUp",
         "type" : "boolean"
      }
   ],
   "default_bindings" : [
      {
+        "binding_url" : "application_generated_unity_retryoverlay_exe_binding_oculus_touch.json",
+        "controller_type" : "oculus_touch"
      }
   ],
   "localization" : []
}

The default binding file is generated into the same directory to actions.json. These files will be included in the build for users.

Click the <- BACK button at the upper left, then you see the new setting is active.

Now we’ve made the default binding. Close the SteamVR Input window.

Also, close the Unity SteamVR Input window.
At this time, it asks you to save the change, so click Close.

In this tutorial environment, clicking the Save button overwrites the action manifest as an empty default_bindings.
(There is no problem with clicking Save when we open the binding setting window next time.)

How about other controller bindings?

You can create other controller’s default bindings by clicking the controller name on the right side.

Other controllers’ default bindings are added to default_bindings param of action manifest.

{
    "action_sets" : [
       {
          "name" : "/actions/NewSet",
          "usage" : "leftright"
       }
    ],
    "actions" : [
       {
          "name" : "/actions/NewSet/in/NewAction",
          "type" : "boolean"
       }
    ],
    "default_bindings" : [
      {
         "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_oculus_touch.json",
         "controller_type" : "oculus_touch"
      },
+     {
+        "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_vive_controller.json",
+        "controller_type" : "vive_controller"
+     }
   ],
   "localization" : []
}

You don’t necessarily have to create other controller default bindings because SteamVR automatically remaps actions for other controllers if at least one default binding exists.

Create script

Create a new script InputController.cs into Scripts folder.
We will add the controller input related code to this file.

On hierarchy, right click > Create Empty to create an empty object, change the object name to InputController.
Drag InputController.cs from the project window to the InputController object.

Initialize and cleanup OpenVR

Copy the below code to InputController.cs.

using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Set action manifest path

Set the action manifest path with SetActionManifestPath() at launch. (read the wiki for details)
The action manifest is generated as StreamingAssets/SteamVR/actions.json so we set this.

using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

+       var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to set action manifest path: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Get action set handle

An application can have some action sets that have multiple actions. We use an action set handle to determine which action set is used.
Get action set handle with GetActionSetHandle().

The action set is set with a string /actions/[action_set_name].
This time, it’s /actions/Watch.

public class InputController : MonoBehaviour
{
+   ulong actionSetHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to set action manifest path: " + error);
        }

+       error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to get action set /actions/Watch: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Get action handle

Next, get the action handle with GetActionHandle().
Action is set with /actions/[action_set_name]/in/[action_name].
This time, action is /actions/Watch/in/Wakeup.

public class InputController : MonoBehaviour
{
    ulong actionSetHandle = 0;
+   ulong actionHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Faild to set action manifest path: " + error);
        }

        error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get action set /actions/Watch: " + error);
        }

+       error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Faild to get action /actions/Watch/in/WakeUp: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Update action state

Update each action state at the first of each frame.

Prepare action set to update

We use an action set to specify which actions will be updated.
Pass target action sets as an array of VRActiveActionSet_t type.
This time, the action set is only /actions/Watch so we make a single-element array VRActionActiveSet_t[].

private void Start()
{
    OpenVRUtil.System.InitOpenVR();

    var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to set action manifest path: " + error);
    }

    error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action set /actions/Watch: " + error);
    }

    error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action /actions/Watch/in/WakeUp: " + error);
    }
}

+ private void Update()
+ {
+     // Action set list to update
+     var actionSetList = new VRActiveActionSet_t[]
+     {
+         // This time, Watch action only.
+         new VRActiveActionSet_t()
+         {
+             // Pass Watch action set handle
+             ulActionSet =  actionSetHandle,
+             ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
+         }
+     };
+ }

...

ulRestrictedToDevice limits input only from a specific device. For exmaple, to bind different actions for left and right controllers. We don't use this basically so set k_ulInvalidInputValueHandle.

Update state

We use UpdateActionState() to update action state. Pass the action set array to UpdateActionState().

private void Update()
{ 
   var actionSetList = new VRActiveActionSet_t[]
   {
       new VRActiveActionSet_t()
       {
           ulActionSet =  actionSetHandle,
           ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
       }
   };

+   var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));    
+   var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
+   if (error != EVRInputError.None)
+   {
+       throw new Exception("Faild to update action state: " + error);
+   }
}

The second argument activeActionSize is the byte size of VRActiveActionSet_t the struct.

Get action value

We have updated the action state. Next, we will get the action value.
The function used to retrieve action value varies depending on the action type.

GetDigitalActionData()

This gets an on/off value likes the button is pushed or not.

GetAnalogActionData()

This gets analog values like thumb stick direction or how much trigger is pulled.

GetPoseActionDataRelativeToNow()

GetPoseActionDataForNextFrame()

This gets a pose like controller position and rotation.

This time, we want to get the button on/off value so use GetDigitalActionData().
Get the action value with WakeUp action handle we prepared.

private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };

    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }

+   var result = new InputDigitalActionData_t();
+   var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
+   error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
+   if (error != EVRInputError.None)
+   {
+       throw new Exception("Failed to get WakeUp action data: " + error);
+   }
}

The 3rd argument digitalActionSize is the byte size of InputDigitalActionData_t structure.
The 4th argument ulRestrictToDevice is generally not used so set OpenVR.k_ulInvalidInputValueHandle.

The returned value type is InputDigitalActionData_t.

struct InputDigitalActionData_t
{
    bool bActive;
    VRInputValueHandle_t activeOrigin;
    bool bState;
    bool bChanged;
    float fUpdateTime;
};

The action on/off state is set to bState.
bChanged is true at the frame where the action state is changed.

Let’s detect the action.

private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };

    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }

    var result = new InputDigitalActionData_t();
    var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
    error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get WakeUp action data: " + error);
    }

+   if (result.bState && result.bChanged)
+   {
+       Debug.Log("WakeUp is executed");
+   }
}

Run the program and push the Y button, then the message is logged to the Unity console.

Create event handler

Notify to Unity Event

Add UnityEvent into InputController.cs to notify WakeUp action, then call it with Invoke().

using System;
using UnityEngine;
using Valve.VR;
+ using UnityEngine.Events;

public class InputController : MonoBehaviour
{
+   public UnityEvent OnWakeUp; 

    private ulong actionSetHandle = 0;
    private ulong actionHandle = 0;

    // code omit.

    private void Update()
    {
        var actionSetList = new VRActiveActionSet_t[]
        {
            new VRActiveActionSet_t()
            {
                ulActionSet =  actionSetHandle,
                ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
            }
        };

        var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
        var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to update action state: " + error);
        }

        var result = new InputDigitalActionData_t();
        var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
        error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get WakeUp action data: " + error);
        }

        if (result.bState && result.bChanged)
        {
-           Debug.Log("WakeUp is executed");
+           OnWakeUp.Invoke();
        }
    }

Attach event handler

Add method into WatchOverlay.cs that will be called when WakeUp action is executed.

public class WatchOverlay : MonoBehaviour
{

    ...

+   public void OnWakeUp()
+   {
+       // Show watch.
+   }
}

In the hierarchy, open InputController inspector, then set WatchOverlay.OnWakeUp() to OnWakeUp() field.

Add code to show/hide the watch

Update WatchOverlay.cs to show the watch only when WakeUp action is executed.

Hide watch by default

private void Start()
{
    OpenVRUtil.System.InitOpenVR();
    overlayHandle = Overlay.CreateOverlay("WatchOverlayKey", "WatchOverlay");

    Overlay.FlipOverlayVertical(overlayHandle);
    Overlay.SetOverlaySize(overlayHandle, size);
-   Overlay.ShowOverlay(overlayHandle);
}

Show watch by WakeUp action

Show overlay when the WakeUp action is executed.

public void OnWakeUp()
{
+   Overlay.ShowOverlay(overlayHandle);
}

Add hide method

Add HideOverlay() to OpenVRUtil.cs.

public static void ShowOverlay(ulong handle)
{
    var error = OpenVR.Overlay.ShowOverlay(handle);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to show overlay: " + error);
    }
}

+ public static void HideOverlay(ulong handle)
+ {
+     var error = OpenVR.Overlay.HideOverlay(handle);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to hide overlay: " + error);
+     }
+ }

public static void SetOverlayFromFile(ulong handle, string path)
{
    var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to draw image file: " + error);
    }
}

Hide watch after three seconds

This time, we use Unity Coroutine to wait three seconds.

WatchOverlay.cs

using System;
+ using System.Collections;
using UnityEngine;
using Valve.VR;
using OpenVRUtil;

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public ETrackedControllerRole targetHand = ETrackedControllerRole.RightHand;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
+   private Coroutine sleepCoroutine;

    ...

    public void OnWakeUp()
    {
        Overlay.ShowOverlay(overlayHandle);
+       if (sleepCoroutine != null)
+       {
+           StopCoroutine(sleepCoroutine);
+       }
+       sleepCoroutine = StartCoroutine(Sleep());
    }

+   private IEnumerator Sleep()
+   {
+       yield return new WaitForSeconds(3);
+       Overlay.HideOverlay(overlayHandle);
+   }
}

Run the program, and check the watch is shown for three seconds when the Y button is pushed.

Complete 🎉

The watch application is completed.
This tutorial ends here, so the final code organization is up to you.

Look back at what we touched on in this tutorial.

Initialize OpenVR
Create overlay
Draw image file
Change overlay size and position
Follow devices
Draw camera output
Create dashboard overlay
Process events
Controller input

We learned the basics of overlay application development.
OpenVR has many more APIs, so how about trying to make original tools?

The next page is additional information for more details.

Next: Appendix

Part 10 — Controller Input

kurohuku — Tue, 18 Jun 2024 15:01:34 +0000

In this part, we will make the watch hidden by default and show it in a few seconds by pressing a controller button.

There are various ways to get controller input with Unity. This time, we will use OpenVR Input API (SteamVR Input).

Prerequisite setting

Open the SteamVR Settings.

Toggle Advanced Settings.

Toggle Developer > Enable debugging options in the input binding user interface to On.

Create action manifest

First, create the action manifest that defines actions in JSON format. We use the Unity SteamVR Plugin GUI tool to generate the action manifest easily.

Generate action manifest

Select Unity menu Window > SteamVR Input.

It asks you whether you want to use a sample action manifest file.
This time we will make up from the first so select No.

Input the action sets name to “Watch”, and select the below dropdown to per hand. An application can have multiple action sets.

It will generate a new action manifest file as StreamingAssets/SteamVR/actions.json.

{
  "actions": [
    {
      "name": "/actions/Watch/in/WakeUp",
      "type": "boolean"
    }
  ],
  "action_sets": [
    {
      "name": "/actions/Watch",
      "usage": "leftright"
    }
  ],
  "default_bindings": [],
  "localization": []
}

Create default binding

Click the right bottom Open binding UI button.

If any VR controller is not shown, check your HMD is recognized from SteamVR.

Click the “Create New Binding”.

Set the Y button to show the watch for a few seconds. If your controller doesn’t have the Y button, use another button instead.

Click the + button to the right of the Y Button.

Select BUTTON.

Select None to the right of the Click, then select wakeup action.

Click the left bottom check icon to save changes.

Click the right bottom Replace Default Binding.

Click Save.

At this point, a new default binding file is added to the action manifest.

actions.json

{
   "action_sets" : [
      {
         "name" : "/actions/Watch",
         "usage" : "single"
      }
   ],
   "actions" : [
      {
         "name" : "/actions/Watch/in/WakeUp",
         "type" : "boolean"
      }
   ],
   "default_bindings" : [
      {
+        "binding_url" : "application_generated_unity_retryoverlay_exe_binding_oculus_touch.json",
+        "controller_type" : "oculus_touch"
      }
   ],
   "localization" : []
}

The default binding file is generated into the same directory to actions.json. These files will be included in the build for users.

Click the <- BACK button at the upper left, then you see the new setting is active.

Now we’ve made the default binding. Close the SteamVR Input window.

Also, close the Unity SteamVR Input window.
At this time, it asks you to save the change, so click Close.

How about other controller bindings?

You can create other controller’s default bindings by clicking the controller name on the right side.

Other controllers’ default bindings are added to default_bindings param of action manifest.

{
    "action_sets" : [
       {
          "name" : "/actions/NewSet",
          "usage" : "leftright"
       }
    ],
    "actions" : [
       {
          "name" : "/actions/NewSet/in/NewAction",
          "type" : "boolean"
       }
    ],
    "default_bindings" : [
      {
         "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_oculus_touch.json",
         "controller_type" : "oculus_touch"
      },
+     {
+        "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_vive_controller.json",
+        "controller_type" : "vive_controller"
+     }
   ],
   "localization" : []
}

You don’t necessarily have to create other controller default bindings because SteamVR automatically remaps actions for other controllers if at least one default binding exists.

Create script

Create a new script InputController.cs into Scripts folder.
We will add the controller input related code to this file.

Initialize and cleanup OpenVR

Copy the below code to InputController.cs.

using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Set action manifest path

Set the action manifest path with SetActionManifestPath() at launch. (read the wiki for details)
The action manifest is generated as StreamingAssets/SteamVR/actions.json so we set this.

using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

+       var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to set action manifest path: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Get action set handle

An application can have some action sets that have multiple actions. We use an action set handle to determine which action set is used.
Get action set handle with GetActionSetHandle().

The action set is set with a string /actions/[action_set_name].
This time, it’s /actions/Watch.

public class InputController : MonoBehaviour
{
+   ulong actionSetHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to set action manifest path: " + error);
        }

+       error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to get action set /actions/Watch: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Get action handle

Next, get the action handle with GetActionHandle().
Action is set with /actions/[action_set_name]/in/[action_name].
This time, action is /actions/Watch/in/Wakeup.

public class InputController : MonoBehaviour
{
    ulong actionSetHandle = 0;
+   ulong actionHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Faild to set action manifest path: " + error);
        }

        error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get action set /actions/Watch: " + error);
        }

+       error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Faild to get action /actions/Watch/in/WakeUp: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

Update action state

Update each action state at the first of each frame.

Prepare action set to update

We use an action set to specify which actions will be updated.
Pass target action sets as an array of VRActiveActionSet_t type.
This time, the action set is only * so we make a single-element array VRActionActiveSet_t[].

private void Start()
{
    OpenVRUtil.System.InitOpenVR();

    var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to set action manifest path: " + error);
    }

    error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action set /actions/Watch: " + error);
    }

    error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action /actions/Watch/in/WakeUp: " + error);
    }
}

+ private void Update()
+ {
+     // Action set list to update
+     var actionSetList = new VRActiveActionSet_t[]
+     {
+         // This time, Watch action only.
+         new VRActiveActionSet_t()
+         {
+             // Pass Watch action set handle
+             ulActionSet =  actionSetHandle,
+             ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
+         }
+     };
+ }

...

The second argument activeActionSize is the byte size of VRActiveActionSet_t the struct.

Get action value

We have updated the action state. Next, we will get the action value.
The function used to retrieve action value varies depending on the action type.

GetDigitalActionData()

This gets an on/off value likes the button is pushed or not.

GetAnalogActionData()

This gets analog values like thumb stick direction or how much trigger is pulled.

GetPoseActionData()

This gets a pose like controller position and rotation.

This time, we want to get the button on/off value so use GetDigitalActionData().
Get the action value with WakeUp action handle we prepared.

private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };

    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }

+   var result = new InputDigitalActionData_t();
+   var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
+   error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
+   if (error != EVRInputError.None)
+   {
+       throw new Exception("Failed to get WakeUp action data: " + error);
+   }
}

The 3rd argument digitalActionSize is the byte size of InputDigitalActionData_t structure.
The 4th argument ulRestrictToDevice is generally not used so set OpenVR.k_ulInvalidInputValueHandle.

The returned value type is InputDigitalActionData_t.

struct InputDigitalActionData_t
{
    bool bActive;
    VRInputValueHandle_t activeOrigin;
    bool bState;
    bool bChanged;
    float fUpdateTime;
};

The action on/off state is set to bState.
bChanged is true at the frame where the action state is changed.

Let’s detect the action.

private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };

    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }

    var result = new InputDigitalActionData_t();
    var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
    error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get WakeUp action data: " + error);
    }

+   if (result.bState && result.bChanged)
+   {
+       Debug.Log("WakeUp is executed");
+   }
}

Run the program and push the Y button, then the message is logged to the Unity console.

Create event handler

Notify to Unity Event

Add UnityEvent into InputController.cs to notify WakeUp action, then call it with Invoke().

using System;
using UnityEngine;
using Valve.VR;
+ using using UnityEngine.Events;

public class InputController : MonoBehaviour
{
+   public UnityEvent OnWakeUp; 

    private ulong actionSetHandle = 0;
    private ulong actionHandle = 0;

    // code omit.

    private void Update()
    {
        var actionSetList = new VRActiveActionSet_t[]
        {
            new VRActiveActionSet_t()
            {
                ulActionSet =  actionSetHandle,
                ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
            }
        };

        var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
        var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to update action state: " + error);
        }

        var result = new InputDigitalActionData_t();
        var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
        error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get WakeUp action data: " + error);
        }

        if (result.bState && result.bChanged)
        {
-           Debug.Log("WakeUp is executed");
+           OnWakeUp.Invoke();
        }
    }

Attach event handler

Add method into WatchOverlay.cs that will be called when WakeUp action is executed.

public class WatchOverlay : MonoBehaviour
{

    ...

+   public void OnWakeUp()
+   {
+       // Show watch.
+   }
}

In the hierarchy, open InputController inspector, then set WatchOverlay.OnWakeUp() to OnWakeUp() field.

Add code to show/hide the watch

Update WatchOverlay.cs to show the watch only when WakeUp action is executed.

Hide watch by default

private void Start()
{
    OpenVRUtil.System.InitOpenVR();
    overlayHandle = Overlay.CreateOverlay("WatchOverlayKey", "WatchOverlay");

    Overlay.FlipOverlayVertical(overlayHandle);
    Overlay.SetOverlaySize(overlayHandle, size);
-   Overlay.ShowOverlay(overlayHandle);
}

Show watch by WakeUp action

Show overlay when the WakeUp action is executed.

public void OnWakeUp()
{
+   Overlay.ShowOverlay(overlayHandle);
}

Add hide method

Add HideOverlay() to OpenVRUtil.cs.

public static void ShowOverlay(ulong handle)
{
    var error = OpenVR.Overlay.ShowOverlay(handle);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to show overlay: " + error);
    }
}

+ public static void HideOverlay(ulong handle)
+ {
+     var error = OpenVR.Overlay.HideOverlay(handle);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to hide overlay: " + error);
+     }
+ }

public static void SetOverlayFromFile(ulong handle, string path)
{
    var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to draw image file: " + error);
    }
}

Hide watch after three seconds

This time, we use Unity Coroutine to wait three seconds.

WatchOverlay.cs

using System;
+ using System.Collections;
using UnityEngine;
using Valve.VR;
using OpenVRUtil;

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public ETrackedControllerRole targetHand = ETrackedControllerRole.RightHand;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
+   private Coroutine sleepCoroutine;

    ...

    public void OnWakeUp()
    {
        Overlay.ShowOverlay(overlayHandle);
+       if (sleepCoroutine != null)
+       {
+           StopCoroutine(sleepCoroutine);
+       }
+       sleepCoroutine = StartCoroutine(Sleep());
    }

+   private IEnumerator Sleep()
+   {
+       yield return new WaitForSeconds(3);
+       Overlay.HideOverlay(overlayHandle);
+   }
}

Run the program, and check the watch is shown for three seconds when the Y button is pushed.

Complete 🎉

The watch application is completed.
This tutorial ends here, so the final code organization is up to you.

Look back at what we touched on in this tutorial.

Initialize OpenVR
Create overlay
Draw image file
Change overlay size and position
Follow devices
Draw camera output
Create dashboard overlay
Process events
Controller input

We learned the basics of overlay application development.
OpenVR has many more APIs, so how about trying to make original tools?

The next page is additional information for more details.

SteamVR Overlay with Unity: Overlay Events

kurohuku — Tue, 18 Jun 2024 13:57:23 +0000

Show the current time on the right hand

Let’s display the current time on the right hand.

Create hand variable

Add a variable in WatchOverlay.cs to determine which hand to show the current time.
OpenVR has ETrackedControllerRole type representing the left or right hand.

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
+   public ETrackedControllerRole targetHand = EtrackedControllerRole.LeftHand;

    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    ...

The left hand is EtrackedControllerRole.LeftHand, and the right hand is EtrackedControllerRole.RightHand.

Get selected controller device index

Currently, it gets only the left controller device index. Let’s make it to get the current selected controller.

private void Update()
{
    var position = new Vector3(x, y, z);
    var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);

-   var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
-   if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
-   {
-       Overlay.SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
-   }
+   var controllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(targetHand);
+   if (controllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
+   {
+       Overlay.SetOverlayTransformRelative(overlayHandle, controllerIndex, position, rotation);
+   }

    Overlay.SetOverlayRenderTexture(overlayHandle, renderTexture);
}

Run the program, switch Target Hand to Right Hand in the WatchOverlay inspector. It should display the current time on the right hand.

But the current time is displayed on the wrong position on the right hand because we set the position for only the left hand.

Let’s fix the position for the right hand.

Note down current params

At first, open WatchOverlay inspector, note down X, Y, Z, RotationX, Rotation Y, Rotation Z values.

Left hand params
x = -0.044
y = 0.015
z = -0.131
rotationX = 154
rotationY = 262
rotationZ = 0

Create variables for each hand

Remove the member variables below.

x, y, z
rotationX, rotationY, rotationZ

Keep the size because it will be shared from both hands.

Add left-hand and right-hand variables below.

leftX, leftY, leftZ
leftRotationX, leftRotationY, leftRotationZ
rightX, rightY, rightZ
rightRotationX, rightRotationY, rightRotationZ

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public ETrackedControllerRole targetHand = ETrackedControllerRole.RightHand;

    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    // Keep size
    [Range(0, 0.5f)] public float size;

-   // Remove current variables
-   [Range(-0.2f, 0.2f)] public float x;
-   [Range(-0.2f, 0.2f)] public float y;
-   [Range(-0.2f, 0.2f)] public float z;
-   [Range(0, 360)] public int rotationX;
-   [Range(0, 360)] public int rotationY;
-   [Range(0, 360)] public int rotationZ;

+   // Add for left hand
+   [Range(-0.2f, 0.2f)] public float leftX;
+   [Range(-0.2f, 0.2f)] public float leftY;
+   [Range(-0.2f, 0.2f)] public float leftZ;
+   [Range(0, 360)] public int leftRotationX;
+   [Range(0, 360)] public int leftRotationY;
+   [Range(0, 360)] public int leftRotationZ;

+   // Add for right hand
+   [Range(-0.2f, 0.2f)] public float rightX;
+   [Range(-0.2f, 0.2f)] public float rightY;
+   [Range(-0.2f, 0.2f)] public float rightZ;
+   [Range(0, 360)] public int rightRotationX;
+   [Range(0, 360)] public int rightRotationY;
+   [Range(0, 360)] public int rightRotationZ;

Switch position and rotation to match the current hand

Edit Update() to set corresponding to targetHandposition and rotation.

private void Update()
{
-   var position = new Vector3(x, y, z);
-   var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
+   Vector3 position;
+   Quaternion rotation;
+       
+   if (targetHand == ETrackedControllerRole.LeftHand)
+   {
+       position = new Vector3(leftX, leftY, leftZ);
+       rotation = Quaternion.Euler(leftRotationX, leftRotationY, leftRotationZ);
+   }
+   else
+   {
+       position = new Vector3(rightX, rightY, rightZ);
+       rotation = Quaternion.Euler(rightRotationX, rightRotationY, rightRotationZ);
+   }

    var controllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(targetHand);
    if (controllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        Overlay.SetOverlayTransformRelative(overlayHandle, controllerIndex, position, rotation);
    }

    Overlay.SetOverlayRenderTexture(overlayHandle, renderTexture);
}

Adjust position for right hand

Run the program and open WatchOverlay inspector.
Check Target Hand is set to Right Hand.

Move the right-hand sliders to adjust the position and rotation to the best fit.

Here are the sample values for the right hand.

Right hand params
x = 0.04
y = 0.003
z = -0.107
rotationX = 24
rotationY = 258
rotationZ = 179

After you set the best position on the sliders, right click theWatchOverlay component name on the inspector > Copy Component to copy values.

Stop the program, right click > Paste Component Values.
Also, input the left-hand values from the note you took.
Reset Target Hand variable to Left Hand.

Here, we got the position and rotation for both hands.

Create button events

Let’s make it so that when the button is clicked, switch controllers.
Create a new script WatchSettingController.cs inside Scripts folder.
We will add the code of button events here.

Right click hierarchy > Create Empty to make an empty object, and change the object name to SettingController.
Add WatchSettingController.cs to the SettingController object.

Copy the code below to WatchSettingController.cs.

using UnityEngine;

public class WatchSettingController : MonoBehaviour
{
    [SerializeField] private WatchOverlay watchOverlay;
}

Add an event when the button is clicked to WatchSettingController.cs.

using UnityEngine;
+ using Valve.VR;

public class WatchSettingController : MonoBehaviour
{
    [SerializeField] private WatchOverlay watchOverlay;

+   public void OnLeftHandButtonClick()
+   {
+       // When "Left Hand" button is clicked, switch to left hand.
+       watchOverlay.targetHand = ETrackedControllerRole.LeftHand;
+   }
+
+   public void OnRightHandButtonClick()
+   {
+       // When "Right Hand" button is clicked, switch to right hand.
+       watchOverlay.targetHand = ETrackedControllerRole.RightHand;
+   }
}

Set button events

Select Dashboard > Canvas > LeftHandButton object on the hierarchy.
On the inspector, click + button in the OnClick() field of Button component.
Drag SettingController object to None (Object) of OnClick() field, and select WatchSettingController.OnLeftHandButtonClick().

Here, SettingController.OnLeftHandButtonClick() is called when the button is clicked.

Similarly, set WatchSettingController.OnRightHandButtonClick() to OnClick() field in RightHandButton inspector.

Get dashboard event

Currently, nothing happens when the button is clicked.
The previous settings are for Unity side events, not OpenVR.

OpenVR and Unity events are different, so we must notify Unity of the OpenVR dashboard event.

First, we get the OpenVR click event.
Overlay events can be detected by polling with PollNextOverlayEvent(). (read the wiki for details)

Let’s watch dashboard overlay events in Update() with PollNextOverlayEvents().
If some overlay events occur on the specified overlay, PollNextOverlayEvent() returns true and takes one event from the event queue. If all events are taken, it returns false.

Add event detection code into Update() of DashboardOverlay.cs.

void Update()
{
    Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);

+   var vrEvent = new VREvent_t();
+   var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));
+
+   // If overlay events are left then true.
+   while (OpenVR.Overlay.PollNextOverlayEvent(dashboardHandle, ref vrEvent, uncbVREvent))
+   {
+       // vrEvent is popped up event.
+   }
+
+   // Break the loop when all events are popped out.
}

OpenVR event represents as VREvent_t type. (read the wiki for details)
uncbVREvent is the byte size of VREvent_t type.

Get EVREventType.VREvent_MouseButtonDown and EVREventType.VREvent_MouseButtonUp events.

private void Update()
{
    var vrEvent = new VREvent_t();
    var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));

    while (OpenVR.Overlay.PollNextOverlayEvent(overlayHandle, ref vrEvent, uncbVREvent))
    {
+       switch ((EVREventType)vrEvent.eventType)
+       {
+           case EVREventType.VREvent_MouseButtonDown:
+               Debug.Log("MouseDown");
+               break;
+
+           case EVREventType.VREvent_MouseButtonUp:
+               Debug.Log("MouseUp");
+               break;
+       }
    }
}

vrEvent.eventType is a uint event code, so we cast it EVREventType for comparison.
Run the program, click the dashboard button in VR, and then “MouseDown” and “MouseUp” should be logged.

Aim with the laser pointer then click!

Is there EVREventType.VREvent_MouseClick?

No there is.
OpenVR Overlay mouse events are simple. There are only three events MouseButtonDown, MouseButtonUp, MouseMove. If you want to use other events like OnClick, OnMouseEnter or OnMouseLeave, you should combine the three basic events. It is harder than Unity UI to make some UIs like slider.

Is there easy way to make UI?

I released a UI asset that has basic UIs for SteamVR dashboard. If you are interested, try it.
https://assetstore.unity.com/packages/tools/gui/ovrle-ui-dashboard-ui-kit-for-steamvr-270636

Get the click position from the overlay mouse event.

private void Update()
{
    var vrEvent = new VREvent_t();
    var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));

    while (OpenVR.Overlay.PollNextOverlayEvent(overlayHandle, ref vrEvent, uncbVREvent))
    {
        switch (vrEvent.eventType)
        {
            case (uint)EVREventType.VREvent_MouseButtonDown:
-               Debug.Log("MouseDown");            
+               Debug.Log($"MouseDown: ({vrEvent.data.mouse.x}, {vrEvent.data.mouse.y})");
                break;

            case (uint)EVREventType.VREvent_MouseButtonUp:
-               Debug.Log("MouseUp");
+               Debug.Log($"MouseUp: ({vrEvent.data.mouse.x}, {vrEvent.data.mouse.y})");
                break;
        }
    }
}

vrEvent has a mouse position as VREvent_Mouse_t type.
The mouse position is UV, so the x and y are 0–1.

Run the program, open the SteamVR dashboard, and click the overlay.
The clicked mouse position should be logged.

Apply mouse scaling factor

The mouse position is 0–1 UV, but we can convert it to the actual UI position (px) by applying the Mouse Scaling Factor with SetOverlayMouseScale(). (read the wiki for details)

Set mouse scaling factor in Start() of DashboardOverlay.cs.

private void Start()
{
    OpenVRUtil.System.InitOpenVR();

    (dashboardHandle, thumbnailHandle) = Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    Overlay.SetOverlayFromFile(thumbnailHandle, filePath);

    Overlay.FlipOverlayVertical(dashboardHandle);
    Overlay.SetOverlaySize(dashboardHandle, 2.5f);

+   var mouseScalingFactor = new HmdVector2_t()
+   {
+       v0 = renderTexture.width,
+       v1 = renderTexture.height
+   };
+   error = OpenVR.Overlay.SetOverlayMouseScale(dashboardHandle, ref mouseScalingFactor);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to set mouse scaling factor: " + error);
+   }
}

v0 is width, v1 is height of mouseScalingFactor.
When the OpenVR mouse event is dispatched, the mouse scaling factor multiplies to the UV position to scale it to the actual UI size.

Run the program, and click the dashboard.
The click position should be scaled to (0, 0) ~ (1024, 768).

Test mouse event with Overlay Viewer

FYI, Overlay Viewer can dispatch mouse events.

Run the program, launch Overlay Viewer, and click WatchDashboardKey from the overlay list.
Check the right bottom Mouse Capture and click the preview area, A mouse event will be dispatched and logged to the Unity console.

It is useful to check events without the HMD.

Detect which button is clicked

Detect which Unity button is clicked by the event mouse position.
Unity’s Graphic Raycaster can detect UI components by specific mouse positions.

Get Graphic Raycaster

Add graphicRaycaster variable to DashboardOverlay.cs.

using UnityEngine;
using Valve.VR;
using System;
using OpenVRUtil;
+ using UnityEngine.UI;

public class DashboardOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
+   public GraphicRaycaster graphicRaycaster;
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    ...

Open Dashboard > DashboardOverlay inspector.
Drag Dashboard > Canvas object to the GraphicRaycaster variable.

Add detection method

Add a method to DashboardOverlay.cs which detects a UI component at the specified position.

private void OnDestroy()
{
    OpenVRUtil.System.ShutdownOpenVR();
}

+ private Button GetButtonByPosition(Vector2 position)
+ {
+    // Return button that at position.x, position.y.
+    // If nothing, return null.
+    return null;
+ }

...

In this tutorial, we only use Button component so we take Button only.

Add EventSystem

GraphicRaycaster requires the PointerEventData as an argument. PointerEvent data is defined in EventSystem which controls Unity events.
At first, add a variable to save EventSystem to DashboardOverlay.cs.

using UnityEngine;
using Valve.VR;
using System;
using OpenVRUtil;
using UnityEngine.UI;
+ using UnityEngine.EventSystems;

public class DashboardOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public GraphicRaycaster graphicRaycaster;
+   public EventSystem eventSystem;

    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    ...

Open Dashboard > DashboardOverlay inspector from the hierarchy.
Drag EventSystem object to the Event System variable.

Create PointerEventData

In GetButtonByPosition(), create PointerEventData with EventSystem.

private Button GetButtonByPosition(Vector2 position)
{
+   var pointerEventData = new PointerEventData(eventSystem);
+   pointerEventData.position = position;
    return null;
}

Get Button with GraphicRaycaster

Pass the pointerEventData to GraphicRaycaster.Raycast() to find a Button component on the click position, and return it if it exists.

using UnityEngine;
using Valve.VR;
using System;
using OpenVRUtil;
using UnityEngine.UI;
using UnityEngine.EventSystems;
+ using System.Collections.Generic;

public class DashboardOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public GraphicRaycaster graphicRaycaster;
    public EventSystem eventSystem;

    ...

    private Button GetButtonByPosition(Vector2 position)
    {
        var pointerEventData = new PointerEventData(eventSystem);
        pointerEventData.position = position;

+       // List to save found elements.
+       var raycastResultList = new List<RaycastResult>();
+
+       // Find elements at pointerEventData position then save to raycastList.
+       graphicRaycaster.Raycast(pointerEventData, raycastResultList);
+
+       // Get buttons from saved list.
+       var raycastResult = raycastResultList.Find(element => element.gameObject.GetComponent<Button>());
+
+       // Return null if no button found.
+       if (raycastResult.gameObject == null)
+       {
+           return null;
+       }
+
+       // Otherwise return found button.
+       return raycastResult.gameObject.GetComponent<Button>();
    }

Here, finding a button with a click position is done.

Call detect function

When the OpenVR mouse event is dispatched, retrieve the mouse position and pass it to the GetButtonByPosition() just created above.
There is no mouse click event in the OpenVR overlay event, so we will use MouseDown forclick detection.

void Update()
{
    Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);

    var vrEvent = new VREvent_t();
    var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));
    while (OpenVR.Overlay.PollNextOverlayEvent(dashboardHandle, ref vrEvent, uncbVREvent))
    {
        switch (vrEvent.eventType)
        {
-           // Remove MouseUp event we don't use this time.
-           case (uint)EVREventType.VREvent_MouseButtonDown:
-               Debug.Log($"MouseDown: ({vrEvent.data.mouse.x}, {vrEvent.data.mouse.y})");
-               break;

            case (uint)EVREventType.VREvent_MouseButtonUp:
-               Debug.Log($"MouseUp: ({vrEvent.data.mouse.x}, {vrEvent.data.mouse.y})");
+               var button = GetButtonByPosition(new Vector2(vrEvent.data.mouse.x, vrEvent.data.mouse.y));
+               Debug.Log(button);
                break;
        }
    }
}

Run the program, and click the buttons on the dashboard. The button name should be logged into the Unity console. Currently, it displays the wrong button name but it’s OK for now.
If we click an empty area in the overlay, it should say Null.

Flip the click position vertically

Why the clicked button name is swapped to the other one is that the mouse position Y-axis (multiplied mouse scaling factor to V value) is reversed between OpenVR overlay and Unity canvas.

OpenVR overlay mouse position is notified as the left bottom is (0, 0). This is originally the same as Unity.
But the notified mouse position is flipped vertically this time because we have flipped the V-axis with SetOverlayTextureBounds() when drawing texture to avoid flipping vertically caused by the UV coordinate difference between DirectX and Unity. This texture bounds change affects the notified mouse position.

In this tutorial, we assume that the graphics API is always DirectX, and always flips texture with SetOverlayTextureBounds(), so we will flip the click position V (Y) axis at the event handling.

void Update()
{
    Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);

    var vrEvent = new VREvent_t();
    var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));
    while (OpenVR.Overlay.PollNextOverlayEvent(dashboardHandle, ref vrEvent, uncbVREvent))
    {
        switch (vrEvent.eventType)
        {
            case (uint)EVREventType.VREvent_MouseButtonUp:
+               vrEvent.data.mouse.y = renderTexture.height - vrEvent.data.mouse.y;
                var button = GetButtonByPosition(new Vector2(vrEvent.data.mouse.x, vrEvent.data.mouse.y));
                Debug.Log(button);
                break;
        }
    }
}

Run the program, click the Left Hand and Right Hand buttons, and check the correct button is captured.

Switch controllers by clicking buttons

Now, we have got which button is clicked. Next, switch which hand to display the current time when a button is clicked.
We have created the code to switch hands and attached it to each Button's onClick method, so we will just call it.

DashboardOverlay.cs

void Update()
{
    Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);

    var vrEvent = new VREvent_t();
    var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));
    while (OpenVR.Overlay.PollNextOverlayEvent(dashboardHandle, ref vrEvent, uncbVREvent))
    {
        switch (vrEvent.eventType)
        {
            case (uint)EVREventType.VREvent_MouseButtonUp:
                vrEvent.data.mouse.y = renderTexture.height - vrEvent.data.mouse.y;
                var button = GetButtonByPosition(new Vector2(vrEvent.data.mouse.x, vrEvent.data.mouse.y));
-               Debug.Log(button);
+               if (button != null)
+               {
+                   button.onClick.Invoke();
+               }
                break;
        }
    }
}

Run the program, open the dashboard then click buttons.
It should switch which hand to show the current time.

Here, creating dashboard overlay and switch controllers are done.

Organize code

Mouse Scaling Factor

Move the mouse scaling factor setting as SetOverlayMouseScale().

OpenVRUtil.cs

...

public static void SetOverlayRenderTexture(ulong handle, RenderTexture renderTexture)
{
    if (!renderTexture.IsCreated())
    {
        return;
    }

    var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
    var texture = new Texture_t
    {
        eColorSpace = EColorSpace.Auto,
        eType = ETextureType.DirectX,
        handle = nativeTexturePtr
    };
    var error = OpenVR.Overlay.SetOverlayTexture(handle, ref texture);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to draw texture: " + error);
    }
}

+ public static void SetOverlayMouseScale(ulong handle, int x, int y)
+ {
+     var pvecMouseScale = new HmdVector2_t()
+     {
+         v0 = x,
+         v1 = y
+     };
+     var error = OpenVR.Overlay.SetOverlayMouseScale(handle, ref pvecMouseScale);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to set mouse scale: " + error);
+     }
+ }

...

DashboardOverlay.cs

private void Start()
{
    OpenVRUtil.System.InitOpenVR();

    (dashboardHandle, thumbnailHandle) = Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    Overlay.SetOverlayFromFile(thumbnailHandle, filePath);

    Overlay.SetOverlaySize(dashboardHandle, 2.5f);
    Overlay.FlipOverlayVertical(dashboardHandle);

-   var pvecMouseScale = new HmdVector2_t()
-   {
-       v0 = renderTexture.width,
-       v1 = renderTexture.height
-   };
-   var error = OpenVR.Overlay.SetOverlayMouseScale(dashboardHandle, ref pvecMouseScale);
-   if (error != EVROverlayError.None)
-   {
-       throw new Exception("Failed to set mouse scale: " + error);
-   }
+   Overlay.SetOverlayMouseScale(dashboardHandle, renderTexture.width, renderTexture.height);
}

Event handling

Move the event handling as ProcessOverlayEvents().

DashboardOverlay.cs

void Update()
{
    Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);
-
-   var vrEvent = new VREvent_t();
-   var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));
-   while (OpenVR.Overlay.PollNextOverlayEvent(dashboardHandle, ref vrEvent, uncbVREvent))
-   {
-       switch (vrEvent.eventType)
-       {
-           case (uint)EVREventType.VREvent_MouseButtonUp:
-               vrEvent.data.mouse.y = renderTexture.height - vrEvent.data.mouse.y;
-               var button = GetButtonByPosition(vrEvent.data.mouse.x, vrEvent.data.mouse.y);
-               if (button != null)
-               {
-                   button.onClick.Invoke();
-               }
-               break;
-       }
-  }
+  ProcessOverlayEvents();
}

...

+ private void ProcessOverlayEvents()
+ {
+     var vrEvent = new VREvent_t();
+     var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));
+     while (OpenVR.Overlay.PollNextOverlayEvent(dashboardHandle, ref vrEvent, uncbVREvent))
+     {
+         switch (vrEvent.eventType)
+         {
+             case (uint)EVREventType.VREvent_MouseButtonUp:
+                 vrEvent.data.mouse.y = renderTexture.height - vrEvent.data.mouse.y;
+                 var button = GetButtonByPosition(new Vector2(vrEvent.data.mouse.x, vrEvent.data.mouse.y));
+                 if (button != null)
+                 {
+                     button.onClick.Invoke();
+                 }
+                 break;
+         }
+     }
+ }

private Button GetButtonByPosition(float x, float y)
{
    ...

Final code

WatchOverlay.cs

using UnityEngine;
using Valve.VR;
using System;
using OpenVRUtil;

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public ETrackedControllerRole targetHand = ETrackedControllerRole.LeftHand;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    [Range(0, 0.5f)] public float size;

    [Range(-0.2f, 0.2f)] public float leftX;
    [Range(-0.2f, 0.2f)] public float leftY;
    [Range(-0.2f, 0.2f)] public float leftZ;
    [Range(0, 360)] public int leftRotationX;
    [Range(0, 360)] public int leftRotationY;
    [Range(0, 360)] public int leftRotationZ;

    [Range(-0.2f, 0.2f)] public float rightX;
    [Range(-0.2f, 0.2f)] public float rightY;
    [Range(-0.2f, 0.2f)] public float rightZ;
    [Range(0, 360)] public int rightRotationX;
    [Range(0, 360)] public int rightRotationY;
    [Range(0, 360)] public int rightRotationZ;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();
        overlayHandle = Overlay.CreateOverlay("WatchOverlayKey", "WatchOverlay");

        Overlay.FlipOverlayVertical(overlayHandle);
        Overlay.SetOverlaySize(overlayHandle, size);
        Overlay.ShowOverlay(overlayHandle);
    }

    private void Update()
    {
        Vector3 position;
        Quaternion rotation;

        if (targetHand == ETrackedControllerRole.LeftHand)
        {
            position = new Vector3(leftX, leftY, leftZ);
            rotation = Quaternion.Euler(leftRotationX, leftRotationY, leftRotationZ);
        }
        else
        {
            position = new Vector3(rightX, rightY, rightZ);
            rotation = Quaternion.Euler(rightRotationX, rightRotationY, rightRotationZ);
        }

        var controllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(targetHand);
        if (controllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
        {
            Overlay.SetOverlayTransformRelative(overlayHandle, controllerIndex, position, rotation);
        }

        Overlay.SetOverlayRenderTexture(overlayHandle, renderTexture);
    }

    private void OnApplicationQuit()
    {
        Overlay.DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

DashboardOverlay.cs

using UnityEngine;
using Valve.VR;
using System;
using OpenVRUtil;
using UnityEngine.UI;
using UnityEngine.EventSystems;
using System.Collections.Generic;

public class DashboardOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public GraphicRaycaster graphicRaycaster;
    public EventSystem eventSystem;
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        (dashboardHandle, thumbnailHandle) = Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        Overlay.SetOverlayFromFile(thumbnailHandle, filePath);

        Overlay.FlipOverlayVertical(dashboardHandle);
        Overlay.SetOverlaySize(dashboardHandle, 2.5f);
        Overlay.SetOverlayMouseScale(dashboardHandle, renderTexture.width, renderTexture.height);
    }

    private void Update()
    {
        Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);
        ProcessOverlayEvents();
    }

    private void OnApplicationQuit()
    {
        Overlay.DestroyOverlay(dashboardHandle);
    }

    private void OnDestroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }

    private void ProcessOverlayEvents()
    {
        var vrEvent = new VREvent_t();
        var uncbVREvent = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VREvent_t));
        while (OpenVR.Overlay.PollNextOverlayEvent(dashboardHandle, ref vrEvent, uncbVREvent))
        {
            switch (vrEvent.eventType)
            {
                case (uint)EVREventType.VREvent_MouseButtonUp:
                    vrEvent.data.mouse.y = renderTexture.height - vrEvent.data.mouse.y;
                    var button = GetButtonByPosition(new Vector2(vrEvent.data.mouse.x, vrEvent.data.mouse.y));
                    if (button != null)
                    {
                        button.onClick.Invoke();
                    }
                    break;
            }
        }
    }

    private Button GetButtonByPosition(Vector2 position)
    {
        var pointerEventData = new PointerEventData(eventSystem);
        pointerEventData.position = new Vector2(position.x, position.y);

        var raycastResultList = new List<RaycastResult>();
        graphicRaycaster.Raycast(pointerEventData, raycastResultList);

        var raycastResult = raycastResultList.Find(element => element.gameObject.GetComponent<Button>());
        if (raycastResult.gameObject == null)
        {
            return null;
        }
        return raycastResult.gameObject.GetComponent<Button>();
    }
}

OpenVRUtil.cs

using UnityEngine;
using Valve.VR;
using System;

namespace OpenVRUtil
{
    public static class System
    {
        public static void InitOpenVR()
        {
            if (OpenVR.System != null) return;

            var error = EVRInitError.None;
            OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
            if (error != EVRInitError.None)
            {
                throw new Exception("Failed to initialize OpenVR: " + error);
            }
        }

        public static void ShutdownOpenVR()
        {
            if (OpenVR.System != null)
            {
                OpenVR.Shutdown();
            }
        }
    }

    public static class Overlay
    {
        public static ulong CreateOverlay(string key, string name)
        {
            var handle = OpenVR.k_ulOverlayHandleInvalid;
            var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to create overlay: " + error);
            }

            return handle;
        }

        public static (ulong, ulong) CreateDashboardOverlay(string key, string name)
        {
            ulong dashboardHandle = 0;
            ulong thumbnailHandle = 0;
            var error = OpenVR.Overlay.CreateDashboardOverlay(key, name, ref dashboardHandle, ref thumbnailHandle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to create dashboard overlay: " + error);
            }

            return (dashboardHandle, thumbnailHandle);
        }

        public static void DestroyOverlay(ulong handle)
        {
            if (handle != OpenVR.k_ulOverlayHandleInvalid)
            {
                var error = OpenVR.Overlay.DestroyOverlay(handle);
                if (error != EVROverlayError.None)
                {
                    throw new Exception("Failed to dispose overlay: " + error);
                }
            }
        }

        public static void SetOverlayFromFile(ulong handle, string path)
        {
            var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to draw image file: " + error);
            }
        }

        public static void ShowOverlay(ulong handle)
        {
            var error = OpenVR.Overlay.ShowOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to show overlay: " + error);
            }
        }

        public static void SetOverlaySize(ulong handle, float size)
        {
            var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to set overlay size: " + error);
            }
        }

        public static void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
        {
            var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
            var matrix = rigidTransform.ToHmdMatrix34();
            var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to set overlay position: " + error);
            }
        }

        public static void SetOverlayTransformRelative(ulong handle, uint deviceIndex, Vector3 position, Quaternion rotation)
        {
            var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
            var matrix = rigidTransform.ToHmdMatrix34();
            var error = OpenVR.Overlay.SetOverlayTransformTrackedDeviceRelative(handle, deviceIndex, ref matrix);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to set overlay position: " + error);
            }
        }

        public static void FlipOverlayVertical(ulong handle)
        {
            var bounds = new VRTextureBounds_t
            {
                uMin = 0,
                uMax = 1,
                vMin = 1,
                vMax = 0
            };

            var error = OpenVR.Overlay.SetOverlayTextureBounds(handle, ref bounds);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to texture flip: " + error);
            }
        }

        public static void SetOverlayRenderTexture(ulong handle, RenderTexture renderTexture)
        {
            if (!renderTexture.IsCreated()) return;

            var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
            var texture = new Texture_t
            {
                eColorSpace = EColorSpace.Auto,
                eType = ETextureType.DirectX,
                handle = nativeTexturePtr
            };
            var error = OpenVR.Overlay.SetOverlayTexture(handle, ref texture);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to draw texture: " + error);
            }
        }

        public static void SetOverlayMouseScale(ulong handle, int x, int y)
        {
            var pvecMouseScale = new HmdVector2_t()
            {
                v0 = x,
                v1 = y
            };
            var error = OpenVR.Overlay.SetOverlayMouseScale(handle, ref pvecMouseScale);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to set mouse scale: " + error);
            }
        }
    }
}

WatchSettingController.cs

using UnityEngine;
using Valve.VR;

public class WatchSettingController : MonoBehaviour
{
    [SerializeField] private WatchOverlay watchOverlay;

    public void OnLeftHandButtonClick()
    {
        watchOverlay.targetHand = ETrackedControllerRole.LeftHand;
    }

    public void OnRightHandButtonClick()
    {
        watchOverlay.targetHand = ETrackedControllerRole.RightHand;
    }
}

Here, we made it to deal OpenVR overlay event.
The final task is to display the current time in a few seconds when the controller button is pushed.

Next: Controller Input

SteamVR Overlay with Unity: Dashboard Overlay

kurohuku — Sun, 16 Jun 2024 19:06:40 +0000

Let’s create a setting screen on the SteamVR dashboard. It switches which hand to display the watch overlay.

Create dashboard overlay

Dashboard overlay is an overlay that is displayed on the SteamVR dashboard. We use this overlay as the setting screen.

Create new script

Create DashboardOverlay.cs inside Scripts folder and copy the following code.

using UnityEngine;
using Valve.VR;
using System;

public class DashboardOverlay : MonoBehaviour
{
    private void Start()
    {
    }
}

Put the script into the scene

On hierarchy, right click > Create Empty to create a new game object named DashboardOverlay. Drag DashboardOverlay.cs to the object.

Prepare overlay handles

Dashboard overlay consists of the main overlay and thumbnail overlay. Both overlays have their handle.
The thumbnail overlay is a small overlay at the bottom of the dashboard used to switch between overlays.

The red rectangle is the thumbnail overlay. The large "Right Hand" button is on the main overlay.

Create two variables for the two overlay handles in the DashboardOverlay.cs.

using UnityEngine;
using Valve.VR;
using System;

public class DashboardOverlay : MonoBehaviour
{
+   private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
+   private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;
}

Create dashboard overlay

Create dashboard overlay with CreateDashboardOverlay(). (read the wiki for details)
Here, we set key as "WatchDashboardKey" and name as "Watch Setting".

CreateDashboardOverlay() creates two overlays and sets their handles to the variables of the 3rd and 4th arguments.

Create utility class

We want to run the program but we must initialize the OpenVR before using the API. The initialization code is in WatchOverlay.cs. If DashboardOverlay.cs runs earlier than that initialization, it will result in an error.

There are various ways but this time, we will create a utility to share common code like initializing OpenVR that is called from other classes.

Create new script

Create OpenVRUtil.cs inside Scripts folder. Copy the following code.

using UnityEngine;
using Valve.VR;
using System;

namespace OpenVRUtil
{
    public static class System
    {
    }
}

Move OpenVR initialization

Move the InitOpenVR() from WatchOverlay.cs to OpenVRUtil.cs. Add static to allow access from other external classes.

WatchOverlay.cs


private void OnDestroy()
{
    DestroyOverlay(overlayHandle);
    ShutdownOpenVR();
}

- private void InitOpenVR()
- {
-   if (OpenVR.System != null) return;
-
-   var error = EVRInitError.None;
-   OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
-   if (error != EVRInitError.None)
-   {
-       throw new Exception("Failed to initialize OpenVR: " + error);
-   }
- }

private void ShutdownOpenVR()
{
    if (OpenVR.System != null)
    {
        OpenVR.Shutdown();
    }
}

...

OpenVRUtil.cs

using UnityEngine;
using Valve.VR;
using System;

namespace OpenVRUtil
{
    public static class System
    {
+       // Add as public static method
+       public static void InitOpenVR()
+       {
+           if (OpenVR.System != null) return;
+
+           var error = EVRInitError.None;
+           OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
+           if (error != EVRInitError.None)
+           {
+               throw new Exception("Failed to initialize OpenVR: " + error);
+           }
+       }
    }
}

Move OpenVR cleanup

Similarly, move the ShutdownOpenVR() as a static method.

WatchOverlay.cs

// ...

private void OnDestroy()
{
    DestroyOverlay(overlayHandle);
    ShutdownOpenVR();
}

- private void ShutdownOpenVR()
- {
-     if (OpenVR.System != null)
-     {
-         OpenVR.Shutdown();
-     }
- }

private ulong CreateOverlay(string key, string name)
{
    var handle = OpenVR.k_ulOverlayHandleInvalid;
    var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to dispose OpenVR: " + error);
    }

    return handle;
}

// ...

OpenVRUtil.cs

using UnityEngine;
using Valve.VR;
using System;

namespace OpenVRUtil
{
    public static class System
    {
        public static void InitOpenVR()
        {
            if (OpenVR.System != null) return;

            var initError = EVRInitError.None;
            OpenVR.Init(ref initError, EVRApplicationType.VRApplication_Overlay);
            if (initError != EVRInitError.None)
            {
                throw new Exception("Failed to initialize OpenVR: " + initError);
            }
        }

+       public static void ShutdownOpenVR()
+       {
+           if (OpenVR.System != null)
+           {
+               OpenVR.Shutdown();
+           }
+       }
    }
}

Move overlay methods

Move overlay methods from WatchOverlay.cs to OpenVRUtil.cs because they will be used by other classes later.

Move all overlay methods from CreateOverlay() to SetOverlayRenderTexture() in WatchOverlay.cs.

CreateOverlay()
DestroyOverlya()
SetOverlayFromFile()
ShowOverlay()
SetoverlaySize()
SetOverlayTransformAbsolute()
SetOverlayTransformRelative()
FlipOverlayVertical()
SetOverlayRenderTexture()

WatchOverlay.cs

private void OnDestroy()
{
    DestroyOverlay(overlayHandle);
    OpenVRUtil.System.ShutdownOpenVR();
}

- private ulong CreateOverlay(string key, string name)
- {
-     var handle = OpenVR.k_ulOverlayHandleInvalid;
-     var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
-     if (error != EVROverlayError.None)
-     {
-         throw new Exception("Failed to create overlay: " + error);
-     }
- 
-     return handle;
- }
- 
- ...
- 
- private void SetOverlayRenderTexture(RenderTexture renderTexture)
- {
-     var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
-     var texture = new Texture_t
-     {
-         eColorSpace = EColorSpace.Auto,
-         eType = ETextureType.DirectX,
-         handle = nativeTexturePtr
-     };
-     var error = OpenVR.Overlay.SetOverlayTexture(overlayHandle, ref texture);
-     if (error != EVROverlayError.None)
-     {
-         throw new Exception("Failed to draw texture: " + error);
-     }
- }

Create new static class Overlay to OpenVRUtil.cs and add all the methods as public static method.

OpenVRUtil.cs

namespace OpenVRUtil
{
    public static class System
    {
        public static void InitOpenVR()
        {
            if (OpenVR.System != null) return;

            var initError = EVRInitError.None;
            OpenVR.Init(ref initError, EVRApplicationType.VRApplication_Overlay);
            if (initError != EVRInitError.None)
            {
                throw new Exception("Failed to initialize OpenVR: " + initError);
            }
        }

        public static void ShutdownOpenVR()
        {
            if (OpenVR.System != null)
            {
                OpenVR.Shutdown();
            }
        }
    }

+   public static class Overlay
+   {
+       public static ulong CreateOverlay(string key, string name)
+       {
+           var handle = OpenVR.k_ulOverlayHandleInvalid;
+           var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
+           if (error != EVROverlayError.None)
+           {
+               throw new Exception("Failed to create overlay: " + error);
+           }
+
+           return handle;
+       }
+
+       ...
+
+       public static void SetOverlayRenderTexture(ulong handle, RenderTexture renderTexture)
+       {
+           var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
+           var texture = new Texture_t
+           {
+               eColorSpace = EColorSpace.Auto,
+               eType = ETextureType.DirectX,
+               handle = nativeTexturePtr
+           };
+           var error = OpenVR.Overlay.SetOverlayTexture(handle, ref texture);
+           if (error != EVROverlayError.None)
+           {
+               throw new Exception("Failed to draw texture: " + error);
+           }
+       }
+   }
}

Update method calls in existing code

Change WatchOverlay.cs code to call the overlay methods from the OpenVRUtil instead of WatchOverlay.cs itself.

WatchOverlay.cs

using System;
using UnityEngine;
using Valve.VR;
+ using OpenVRUtil;

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    [Range(0, 0.5f)] public float size;
    [Range(-0.5f, 0.5f)] public float x;
    [Range(-0.5f, 0.5f)] public float y;
    [Range(-0.5f, 0.5f)] public float z;
    [Range(0, 360)] public int rotationX;
    [Range(0, 360)] public int rotationY;
    [Range(0, 360)] public int rotationZ;

    private void Start()
    {
-       InitOpenVR();
+       OpenVRUtil.System.InitOpenVR();

-       overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");
+       overlayHandle = Overlay.CreateOverlay("WatchOverlayKey", "WatchOverlay");

-       FlipOverlayVertical(overlayHandle);
-       SetOverlaySize(overlayHandle, size);
-       ShowOverlay(overlayHandle);
+       Overlay.FlipOverlayVertical(overlayHandle);
+       Overlay.SetOverlaySize(overlayHandle, size);
+       Overlay.ShowOverlay(overlayHandle);
    }

    private void Update()
    {
        var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
        if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
        {
            var position = new Vector3(x, y, z);
            var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
-           SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
+           Overlay.SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
        }

-       SetOverlayRenderTexture(overlayHandle, renderTexture);
+       Overlay.SetOverlayRenderTexture(overlayHandle, renderTexture);
    }

    private void OnApplicationQuit()
    {
-       DestroyOverlay(overlayHandle);
+       Overlay.DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
-       ShutdownOpenVR();
+       OpenVRUtil.System.ShutdownOpenVR();
    }
}

Add OpenVR initialize and cleanup

We made the utility class to share the common code. Let’s go back to the DashboardOverlay.cs to create a dashboard overlay.

Add OpenVR initialization and cleanup code. We already call the initialize and cleanup functions in the WatchOverlay.cs but there is no problem because if the OpenVR is already initialized or cleaned up, the methods do nothing.

DashboardOverlay.cs

using UnityEngine;
using Valve.VR;
using System;
+ using OpenVRUtil;

public class DashboardOverlay : MonoBehaviour
{
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
+       OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting", ref dashboardHandle, ref thumbnailHandle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create dashboard overlay: " + error);
        }
    }

+   private void OnDestroy()
+   {
+       OpenVRUtil.System.ShutdownOpenVR();
+   }
}

Destroy dashboard overlay

Destroy the dashboard overlay at the end of the application.

public class DashboardOverlay : MonoBehaviour
{
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting", ref dashboardHandle, ref thumbnailHandle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create dashboard overlay: " + error);
        }
    }

+   private void OnApplicationQuit()
+   {
+       Overlay.DestroyOverlay(dashboardHandle);
+   }

    private void OnDestroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

As a note, we can destroy the main overlay only. If we pass the thumbnail overlay handle to DestroyOverlay(), ThumbnailCantBeDestroyed error will occur.

Show thumbnail

Let’s show image to the thumbnail.
We made a function SetOverlayFromFile() and a image in part.2 so we will use it.

private void Start()
{
   OpenVRUtil.System.InitOpenVR();

   var error = OpenVR.Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting", ref dashboardHandle, ref thumbnailHandle);
   if (error != EVROverlayError.None)
   {
       throw new Exception("ダッシュボードオーバーレイの作成に失敗しました: " + error);
   }

+  var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
+  Overlay.SetOverlayFromFile(thumbnailHandle, filePath);
}

Run the program, and check if the thumbnail is shown at the bottom of the dashboard.

Destroy the dashboard overlay at the end of the application.

public class DashboardOverlay : MonoBehaviour
{
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting", ref dashboardHandle, ref thumbnailHandle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create dashboard overlay: " + error);
        }
    }

+   private void OnApplicationQuit()
+   {
+       Overlay.DestroyOverlay(dashboardHandle);
+   }

    private void OnDestroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

As a note, we can destroy the main overlay only. If we pass the thumbnail overlay handle to DestroyOverlay(), ThumbnailCantBeDestroyed error will occur.

Show thumbnail

Let's show image to the thumbnail.
We made a function SetOverlayFromFile() and a image in part.2 so we will use it.

private void Start()
{
   OpenVRUtil.System.InitOpenVR();

   var error = OpenVR.Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting", ref dashboardHandle, ref thumbnailHandle);
   if (error != EVROverlayError.None)
   {
       throw new Exception("ダッシュボードオーバーレイの作成に失敗しました: " + error);
   }

+  var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
+  Overlay.SetOverlayFromFile(thumbnailHandle, filePath);
}

Run the program, and check if the thumbnail is shown at the bottom of the dashboard.

We don’t draw any image yet to the main overlay so it shows nothing when the thumbnail is clicked.
The name passed to CreateDashboardOverlay() is shown when the laser pointer hovers over the thumbnail. This time, it’s "Watch Settings".

Create setting screen

Let’s create the setting screen.
First, divide the game objects into two groups: the watch group and the dashboard group.

Watch Group

Right click in hierarchy > Create Empty to create an empty game object named Watch.
Move existing WatchOverlay, Camera, Canvas objects under the Watch object.

Dashboard group

Create the below objects under the Dashboard object.

Camera
UI > Canvas

Camera setting

Select Camera under the Dashboard, set Clear Flags to Solid Color in the inspector.
Click Background color, then change to opaque gray (RGBA = 64, 64, 64, 255).

Also, remove the AudioListener component.

Create render texture

Create render texture for dashboard overlay.

In the project window, right click Assets/RenderTexture folder > Render Texture to create a new render texture asset.
Change the asset name to DashboardRenderTexture.

Set the Size to 1024 x 768 in the inspector.

Click Camera under Dashboard in the hierarchy, drag DashboardRenderTexture asset to Target Texture property.

Here, the dashboard camera output goes to the render texture asset.

Canvas setting

Select Canvas object under Dashboard.
Set Render Mode to Screen Space — Camera in the Canvas inspector.
Drag Camera object under Dashboard to Render Camera.
Set Plane Distance to 10.

Move group

Open Dashboard the object’s inspector.
Set Position X to 20 to avoid overlapping groups.

Create button

Right click Canvas object under Dashboard then create UI > Button — TextMeshPro to create a new button named LeftHandButton.

Click Text (TMP) under LeftHandButton, change the text to “Left Hand”.

Select LeftHandButton and set the Width to 700 and Height to 200.

Select the Text (TMP) and set the Font Size to 100.

Duplicate LeftHandButton by right click > Duplicate.
Change the duplicated object name to RightHandButton.
Similarly, change the button text to “Right Hand”.

Set LeftHandButton Pos Y to 150 and RightHandButton Pos Y to -150.

Here, the appearance of the setting screen is done.

Draw to dashboard

Display the setting screen as a dashboard overlay.

Add camera variable

Add camera member variable to DashboardOverlay.cs.

public class DashboardOverlay : MonoBehaviour
{
+   public Camera camera;
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    // ...

Open DashboardOverlay object inspector, drag Camera object under Dashboard to the Camera variable.

Add render texture

Add renderTexture variable in DashboardOverlay.cs.

public class DashboardOverlay : MonoBehaviour
{
    public Camera camera;
+   public RenderTexture renderTexture; 
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    // ...

Select DashboardOverlay in the hierarchy, drag Assets/RenderTextures/DashboardRenderTexture asset to the Render Texture variable.

Flip vertical and set size

In advance, flip the texture vertically with FlipOverlayVertical() and set overlay size to 2.5 m with SetOverlaySize().

Draw render texture to dashboard overlay

Create Update() and draw the render texture to the dashboard overlay.

DashboardOverlay.cs

public class DashboardOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture; 
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting", ref dashboardHandle, ref thumbnailHandle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create dashboard overlay: " + error);
        }

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        Overlay.SetOverlayFromFile(thumbnailHandle, filePath);

        renderTexture = new RenderTexture(1024, 768, 16, RenderTextureFormat.ARGBFloat);
        camera.targetTexture = renderTexture;

        Overlay.SetOverlaySize(dashboardHandle, 2.5f);
        Overlay.FlipOverlayVertical(dashboardHandle);
    }

+   private void Update()
+   {
+       Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);
+   }

    // ...

Run the program, open the dashboard, and click the thumbnail. The setting screen should be shown.

Organize code

Move the creating dashboard overlay code to CreateDashboardOverlay().
It has to return two values dashboardhandle and thumbnailHandle, so we use a tuple to combine them.

Add the method to the utility’s Overlay class.

OpenVRUtil.cs

...

public static ulong CreateOverlay(string key, string name)
{
    var handle = OpenVR.k_ulOverlayHandleInvalid;
    var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to create overlay: " + error);
    }

    return handle;
}

+ public static (ulong, ulong) CreateDashboardOverlay(string key, string name)
+ {
+     ulong dashboardHandle = 0;
+     ulong thumbnailHandle = 0;
+     var error = OpenVR.Overlay.CreateDashboardOverlay(key, name, ref dashboardHandle, ref thumbnailHandle);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to create dashboard overlay: " + error);
+     }
+ 
+     return (dashboardHandle, thumbnailHandle);
+ }

...

Accordingly, change the function call.

DashboardOverlay.cs

private void Start()
{
    OpenVRUtil.System.InitOpenVR();

-   var error = OpenVR.Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting", ref dashboardHandle, ref thumbnailHandle);
-   if (error != EVROverlayError.None)
-   {
-       throw new Exception("Failed to create dashboard overlay: " + error);
-   }
+   (dashboardHandle, thumbnailHandle) = Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    Overlay.SetOverlayFromFile(thumbnailHandle, filePath);

    Overlay.SetOverlaySize(dashboardHandle, 2.5f);
    Overlay.FlipOverlayVertical(dashboardHandle);
}

Final code

DashboardOverlay.cs

using UnityEngine;
using Valve.VR;
using OpenVRUtil;

public class DashboardOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    private ulong dashboardHandle = OpenVR.k_ulOverlayHandleInvalid;
    private ulong thumbnailHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        (dashboardHandle, thumbnailHandle) = Overlay.CreateDashboardOverlay("WatchDashboardKey", "Watch Setting");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        Overlay.SetOverlayFromFile(thumbnailHandle, filePath);

        Overlay.FlipOverlayVertical(dashboardHandle);
        Overlay.SetOverlaySize(dashboardHandle, 2.5f);
    }

    private void Update()
    {
        Overlay.SetOverlayRenderTexture(dashboardHandle, renderTexture);
    }

    private void OnApplicationQuit()
    {
        Overlay.DestroyOverlay(dashboardHandle);
    }

    private void OnDestroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}

OpenVRUtil.cs

using UnityEngine;
using Valve.VR;
using System;

namespace OpenVRUtil
{
    public static class System
    {
        public static void InitOpenVR()
        {
            if (OpenVR.System != null) return;

            var error = EVRInitError.None;
            OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
            if (error != EVRInitError.None)
            {
                throw new Exception("Failed to initialize OpenVR: " + error);
            }
        }

        public static void ShutdownOpenVR()
        {
            if (OpenVR.System != null)
            {
                OpenVR.Shutdown();
            }
        }
    }

    public static class Overlay
    {
        public static ulong CreateOverlay(string key, string name)
        {
            var handle = OpenVR.k_ulOverlayHandleInvalid;
            var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to create overlay: " + error);
            }

            return handle;
        }

        public static (ulong, ulong) CreateDashboardOverlay(string key, string name)
        {
            ulong dashboardHandle = 0;
            ulong thumbnailHandle = 0;
            var error = OpenVR.Overlay.CreateDashboardOverlay(key, name, ref dashboardHandle, ref thumbnailHandle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to create dashboard overlay: " + error);
            }

            return (dashboardHandle, thumbnailHandle);
        }

        public static void DestroyOverlay(ulong handle)
        {
            if (handle != OpenVR.k_ulOverlayHandleInvalid)
            {
                var error = OpenVR.Overlay.DestroyOverlay(handle);
                if (error != EVROverlayError.None)
                {
                    throw new Exception("Failed to dispose overlay: " + error);
                }
            }
        }

        public static void SetOverlayFromFile(ulong handle, string path)
        {
            var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to draw image file: " + error);
            }
        }

        public static void ShowOverlay(ulong handle)
        {
            var error = OpenVR.Overlay.ShowOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed show overlay: " + error);
            }
        }

        public static void SetOverlaySize(ulong handle, float size)
        {
            var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to set overlay size: " + error);
            }
        }

        public static void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
        {
            var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
            var matrix = rigidTransform.ToHmdMatrix34();
            var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to set overlay position: " + error);
            }
        }

        public static void SetOverlayTransformRelative(ulong handle, uint deviceIndex, Vector3 position, Quaternion rotation)
        {
            var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
            var matrix = rigidTransform.ToHmdMatrix34();
            var error = OpenVR.Overlay.SetOverlayTransformTrackedDeviceRelative(handle, deviceIndex, ref matrix);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to set overlay position: " + error);
            }
        }

        public static void FlipOverlayVertical(ulong handle)
        {
            var bounds = new VRTextureBounds_t
            {
                uMin = 0,
                uMax = 1,
                vMin = 1,
                vMax = 0
            };

            var error = OpenVR.Overlay.SetOverlayTextureBounds(handle, ref bounds);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to flip texture: " + error);
            }
        }

        public static void SetOverlayRenderTexture(ulong handle, RenderTexture renderTexture)
        {
            if (!renderTexture.IsCreated()) return;

            var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
            var texture = new Texture_t
            {
                eColorSpace = EColorSpace.Auto,
                eType = ETextureType.DirectX,
                handle = nativeTexturePtr
            };
            var error = OpenVR.Overlay.SetOverlayTexture(handle, ref texture);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to draw texture: " + error);
            }
        }
    }
}

Now, we have created the setting screen as a dashboard overlay. We will make the button events in the next part.

Next: Overlay Events

SteamVR Overlay with Unity: Draw Camera Output

kurohuku — Sun, 16 Jun 2024 19:06:13 +0000

Prepare camera output

Before displaying the current time to the watch overlay, try displaying the camera output of a simple 3d scene.

Prepare scene

In the hierarchy window, create the following game objects.

Camera
3D Object > Cube
Light > Directional Light

Arrange the objects as the camera captures the cube. It’s OK to arrange roughly.

Rotate the cube

We want to make animated camera output so create a script to rotate the cube. Create Rotate.cs inside the Scripts folder, then copy the following code.

using UnityEngine;

public class Rotate : MonoBehaviour
{
    void Update()
    {
        transform.Rotate(0, 0.5f, 0);
    }
}

Attach the Rotate.cs to the Cube object in the scene.

Run the program and check the cube is rotating.

Here, the test scene setup is done.

Add camera reference

Add a camera member variable to WatchOverlay.cs.

public class WatchOverlay : MonoBehaviour
{
+   public Camera camera;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    [Range(0, 0.5f)] public float size;
    [Range(-0.2f, 0.2f)] public float x;
    [Range(-0.2f, 0.2f)] public float y;
    [Range(-0.2f, 0.2f)] public float z;
    [Range(0, 360)] public int rotationX;
    [Range(0, 360)] public int rotationY;
    [Range(0, 360)] public int rotationZ;

    ...

Set the scene camera to the camera variable of the WatchOverlay object in the Unity inspector.

Remove image file code

To use the camera output, remove the image file code.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

-   var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
-   SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, size);
    ShowOverlay(overlayHandle);
}

Create Render Texture

Create a new RenderTextures folder inside the Assets in the project window. Right click the RenderTextures folder > Create > RenderTexture to create a new render texture asset, and named WatchRenderTexture. We will set the scene camera output to this render texture asset.

Write camera output to the RenderTexture asset

On the hierarchy, click Camera object and open the inspector. Drag WatchRenderTexture from the project window to the camera Target Texture property in the inspector.

Then the camera output is written into WatchRenderTexture asset.

RenderTexture setting

On the project window, click WatchRenderTexture asset and open the inspector. Change Size to 512 x 512.

Optional: When you want to create RenderTexture by script

If you want to create RenderTexture in the code without creating a RenderTexture asset, you can write as below.

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
+   private RenderTexture renderTexture;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    ...

    private void Start()
    {        
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

+       // Set camera.targetTexture to write the camera output to the render texture.
+       renderTexture = new RenderTexture(512, 512, 16, RenderTextureFormat.ARGBFloat);
+       camera.targetTexture = renderTexture;

        SetOverlaySize(overlayHandle, size);
        ShowOverlay(overlayHandle);
    }

    ...

Add Render Texture variable

Add a member variable to store RenderTexture.

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
+   public RenderTexture renderTexture;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    ...

Open WatchOverlay game object inspector, and set WatchRenderTexture asset to renderTexture variable.

Wait for texture creation

Wait for the render texture to be ready to use.

private void Update()
{
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        var position = new Vector3(x, y, z);
        var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
        SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
    }

+   if (!renderTexture.IsCreated())
+   {
+       return;
+   }
+
+   // add render code here
}

Get a native texture pointer

The texture data type to pass to the OpenVR is a graphics API, such as DirectX or OpenGL, which works under the Unity API layer.
We can get a native texture pointer to access the low level API texture data with Unity’s GetNativeTexturePtr().

Call GetNativeTexturePtr() in Update() to get the texture data reference which will be passed to the OpenVR API.

private void Update()
{
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        var position = new Vector3(x, y, z);
        var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
        SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
    }

    if (!renderTexture.IsCreated())
    {
        return;
    }
+   var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
}

Optional: Sync rendering thread

The GetNativeTexturePtr() documentation does not recommend calling it every frame for performance. However, we call it in Update() in this tutorial, because rendering texture to the overlay without syncing to the rendering thread sometimes crashes the program.

Create an OpenVR texture

Create a texture variable of OpenVR texture data type Texture_t.

private void Update()
{
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        var position = new Vector3(x, y, z);
        var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
        SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
    }

    if (!renderTexture.IsCreated())
    {
        return;
    }
    var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
+   var texture = new Texture_t
+   {
+       eColorSpace = EColorSpace.Auto,
+       eType = ETextureType.DirectX,
+       handle = nativeTexturePtr
+   };
}

Set graphics API type to eType. Since DirectX is the default graphics API in this tutorial environment, we are going to assume that the graphics API is DirectX from now on.

Set the native texture pointer to handle.
The camera output is written into this texture.

Optional: If you want to support APIs other than DirectX

You can detect current graphics API with SystemInfo.graphicsDeviceType.

switch (SystemInfo.graphicsDeviceType)
{
    case GraphicsDeviceType.Direct3D11:
        texture.eType = ETextureType.DirectX;
        break;
    case GraphicsDeviceType.Direct3D12:
        texture.eType = ETextureType.DirectX12;
        break;
    case GraphicsDeviceType.OpenGLES2:
    case GraphicsDeviceType.OpenGLES3:
    case GraphicsDeviceType.OpenGLCore:
        texture.eType = ETextureType.OpenGL;
        break;
    case GraphicsDeviceType.Vulkan:
        texture.eType = ETextureType.Vulkan;
        break;
}

You can check the program behavior with APIs other than Direct3D11 by unchecking Project Settings > Player > Other Settings > Auto Graphics API for Windows, and adding the desired graphics API to Graphics APIs for Windows.

https://docs.unity3d.com/Manual/GraphicsAPIs.html

Draw texture to the overlay

Draw texture to the overlay with SetOverlayTexture(). (read the wiki for details)
Pass the Texture_t texture we created before.

rivate void Update()
{
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        var position = new Vector3(x, y, z);
        var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
        SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
    }


    if (!renderTexture.IsCreated())
    {
        return;
    }
    var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
    var texture = new Texture_t
    {
        eColorSpace = EColorSpace.Auto,
        eType = ETextureType.DirectX,
        handle = nativeTexturePtr
    };
+   var error = OpenVR.Overlay.SetOverlayTexture(overlayHandle, ref texture);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to draw texture: " + error);
+   }
}

Run the program. Make sure the camera output is shown on the overlay.

Flip vertical

The camera output is vertically flipped, so we will have to flip it back.

This happens because the V-axis direction of the texture UV system is flipped between Unity and DirectX. Unity’s V-axis faces upwards while DirectX’s faces downwards.

https://docs.unity3d.com/Manual/SL-PlatformDifferences.html

There are different ways to fix this issue. However, we use the OpenVR SetOverlayTextureBounds() to vertically flip the V-axis. (read the wiki for details)

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

+   var bounds = new VRTextureBounds_t
+   {
+       uMin = 0,
+       uMax = 1,
+       vMin = 1,
+       vMax = 0
+   };
+   var error = OpenVR.Overlay.SetOverlayTextureBounds(overlayHandle, ref bounds);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to flip texture: " + error);
+   }

    SetOverlaySize(overlayHandle, size);
    ShowOverlay(overlayHandle);
}

This flips texture vertically. Run the program and check the texture is flipped.

Optional: Support for APIs other than DirectX

In this tutorial, we assume that the graphics API is DirectX so always flip the texture vertically.
If you want to support other APIs, check graphicsDeviceType and deal as “If the graphicsDeviceType is the OpenGL, don’t flip the texture” similar to the above section “Optional: If you want to support APIs other than DirectX”.

Create a canvas to display the current time

We have displayed the camera output to the overlay. Let’s create the watch next.
Remove the Cube and Directional Light, we never use them later.

Open the Camera inspector, and click Reset on the Transform component to reset the position to (0, 0, 0).

Create the below objects into the scene.

UI > Canvas
UI > Text — TextMeshPro inside the Canvas object

When we create TextMeshPro for the first time, it shows the dialog below, click Import TMP Essentials button. Close the dialog after importing.

Open the Canvas inspector, and change Render mode to Screen Space -Camera. Then drag the camera object from the hierarchy to the Render Camera property.

Open the Text (TMP) object inspector, and center the text vertically and horizontally with Alignment.
Input “00:00:00” to the text.

Select the Camera object, and set Clear Flags to Solid Color.
Click the Background color, and make sure A (Alpha) is 0.
This makes the camera background transparent and displays the current time only.

Select the Canvas object, and set the Plane Distance to 10. This makes editing easy on the Editor.

Select Text (TMP), click the anchor setting (the rectangle on the left top of the Rect Transform component), and select the right bottom blue crossed arrow (stretch to both directions).

Set Left, Top, Right, and Bottom to 0.

Scroll the inspector, and set the TextMeshPro — Text (UI) component Font Size to 70.

Run the program. Check if the current time is displayed on the left wrist.
Adjust the position and font size if it is off.

Activate the watch

Create a new script Watch.cs inside the Scripts folder. Copy the following code.

using UnityEngine;
using System;
using TMPro;

public class Watch : MonoBehaviour
{
    private TextMeshProUGUI label;

    void Start()
    {
        label = GetComponent<TextMeshProUGUI>();
    }

    void Update()
    {
        var hour = DateTime.Now.Hour;
        var minute = DateTime.Now.Minute;
        var second = DateTime.Now.Second;
        label.text = $"{hour:00}:{minute:00}:{second:00}";
    }
}

Add Watch.cs to the Text (TMP) object into the scene.

Run the program. It should display the current time.

Organize code

Organize WatchOverlay.cs.

Flip overlay vertically

Move flipping code into FlipOverlayVertical().

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

-   var bounds = new VRTextureBounds_t
-   {
-       uMin = 0,
-       uMax = 1,
-       vMin = 1,
-       vMax = 0
-   };
-   var error = OpenVR.Overlay.SetOverlayTextureBounds(overlayHandle, ref bounds);
-   if (error != EVROverlayError.None)
-   {
-       throw new Exception("Failed to flip texture: " + error);
-   }
+   FlipOverlayVertical(overlayHandle);    
    SetOverlaySize(overlayHandle, size);
    ShowOverlay(overlayHandle);
}

...

+ private void FlipOverlayVertical(ulong handle)
+ {
+    var bounds = new VRTextureBounds_t
+    {
+        uMin = 0,
+        uMax = 1,
+        vMin = 1,
+        vMax = 0
+    };
+
+    var error = OpenVR.Overlay.SetOverlayTextureBounds(handle, ref bounds);
+    if (error != EVROverlayError.None)
+    {
+        throw new Exception("Failed to flip texture: " + error);
+    }
+ }

Draw RenderTexture

Move drawing code into SetOverlayRenderTexture().

private void Update()
{
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        var position = new Vector3(x, y, z);
        var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
        SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
    }

-   var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
-   var texture = new Texture_t
-   {
-       eColorSpace = EColorSpace.Auto,
-       eType = ETextureType.DirectX,
-       handle = nativeTexturePtr
-   };
-   var error = OpenVR.Overlay.SetOverlayTexture(overlayHandle, ref texture);
-   if (error != EVROverlayError.None)
-   {
-       throw new Exception("Failed to draw texture: " + error);
-   }
+   SetOverlayRenderTexture(overlayHandle, renderTexture);
}

...

+ private void SetOverlayRenderTexture(ulong handle, RenderTexture renderTexture)
+ {
+     if (!renderTexture.IsCreated()) return;
+
+     var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
+     var texture = new Texture_t
+     {
+         eColorSpace = EColorSpace.Auto,
+         eType = ETextureType.DirectX,
+         handle = nativeTexturePtr
+     };
+     var error = OpenVR.Overlay.SetOverlayTexture(handle, ref texture);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to draw texture: " + error);
+     }
+ }

Final code

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    [Range(0, 0.5f)] public float size;
    [Range(-0.2f, 0.2f)] public float x;
    [Range(-0.2f, 0.2f)] public float y;
    [Range(-0.2f, 0.2f)] public float z;
    [Range(0, 360)] public int rotationX;
    [Range(0, 360)] public int rotationY;
    [Range(0, 360)] public int rotationZ;

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

        FlipOverlayVertical(overlayHandle);
        SetOverlaySize(overlayHandle, size);
        ShowOverlay(overlayHandle);
    }

    private void Update()
    {
        var position = new Vector3(x, y, z);
        var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
        var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
        if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
        {
            SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
        }

        SetOverlayRenderTexture(overlayHandle, renderTexture);
    }

    private void OnApplicationQuit()
    {
        DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
        ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

    private void ShutdownOpenVR()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }

    private ulong CreateOverlay(string key, string name)
    {
        var handle = OpenVR.k_ulOverlayHandleInvalid;
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create overlay: " + error);
        }

        return handle;
    }

    private void DestroyOverlay(ulong handle)
    {
        if (handle != OpenVR.k_ulOverlayHandleInvalid)
        {
            var error = OpenVR.Overlay.DestroyOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to dispose overlay: " + error);
            }
        }
    }

    private void SetOverlayFromFile(ulong handle, string path)
    {
        var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to draw image file: " + error);
        }
    }

    private void ShowOverlay(ulong handle)
    {
        var error = OpenVR.Overlay.ShowOverlay(handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to show overlay: " + error);
        }
    }

    private void SetOverlaySize(ulong handle, float size)
    {
        var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay size: " + error);
        }
    }

    private void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
    {
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }
    }

    private void SetOverlayTransformRelative(ulong handle, uint deviceIndex, Vector3 position, Quaternion rotation)
    {
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        var error = OpenVR.Overlay.SetOverlayTransformTrackedDeviceRelative(handle, deviceIndex, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }
    }

    private void FlipOverlayVertical(ulong handle)
    {
        var bounds = new VRTextureBounds_t
        {
            uMin = 0,
            uMax = 1,
            vMin = 1,
            vMax = 0
        };

        var error = OpenVR.Overlay.SetOverlayTextureBounds(handle, ref bounds);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to flip texture: " + error);
        }
    }

    private void SetOverlayRenderTexture(ulong handle, RenderTexture renderTexture)
    {
        if (!renderTexture.IsCreated()) return;

        var nativeTexturePtr = renderTexture.GetNativeTexturePtr();
        var texture = new Texture_t
        {
            eColorSpace = EColorSpace.Auto,
            eType = ETextureType.DirectX,
            handle = nativeTexturePtr
        };
        var error = OpenVR.Overlay.SetOverlayTexture(handle, ref texture);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to draw texture: " + error);
        }
    }
}

Finally, we have displayed the current time on the left wrist. In the next part, we will make the dashboard setting screen to switch which hand to display the overlay.

Next: Dashboard Overlay

SteamVR Overlay with Unity: Follow Devices

kurohuku — Sun, 16 Jun 2024 19:05:18 +0000

Follow the HMD

Overlay following the HMD

Remove the position code

First, remove the absolute position code we added in the previous part.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, 0.5f);

-   var position = new Vector3(0, 2, 3);
-   var rotation = Quaternion.Euler(0, 0, 45);
-   SetOverlayTransformAbsolute(overlayHandle, position, rotation);

    ShowOverlay(overlayHandle);

    ...

Device Index

In SteamVR, connected devices are identified with Device Index that automatically allocates from the system. (read the wiki for details)
For HMD, it is defined as OpenVR.k_unTrackedDeviceIndex_Hmd and is always 0.

Prepare position and rotation

Let’s display the overlay at 2 m ahead (Z-axis) of the HMD. As in the previous part, let position and rotation variables.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

+   var position = new Vector3(0, 0, 2);
+   var rotation = Quaternion.Euler(0, 0, 0);

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, 0.5f);
    ShowOverlay(overlayHandle);
}

Set relative position based on the HMD

Use SetOverlayTransformTrackedDeviceRelative() to set the relative position based on the HMD. (read the wiki for details)

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var position = new Vector3(0, 0, 2);
    var rotation = Quaternion.Euler(0, 0, 0);
    var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
    var matrix = rigidTransform.ToHmdMatrix34();
+   var error = OpenVR.Overlay.SetOverlayTransformTrackedDeviceRelative(overlayHandle, OpenVR.k_unTrackedDeviceIndex_Hmd, ref matrix);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to set overlay position: " + error);
+   }

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, 0.5f);
    ShowOverlay(overlayHandle);
}

Pass the HMD device index (OpenVR.k_unTrackedDeviceIndex_Hmd) and the transformation matrix.
Run the program, and check the overlay is shown 2 m ahead of the HMD.

Organize code

Move the relative position code into SetOverlayTransformRelative().

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var position = new Vector3(0, 0, 2);
    var rotation = Quaternion.Euler(0, 0, 0);
-   var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
-   var matrix = rigidTransform.ToHmdMatrix34();
-   var error = OpenVR.Overlay.SetOverlayTransformTrackedDeviceRelative(overlayHandle, OpenVR.k_unTrackedDeviceIndex_Hmd, ref matrix);
-   if (error != EVROverlayError.None)
-   {
-       throw new Exception("Failed to set overlay position: " + error);
-   }
+   SetOverlayTransformRelative(overlayHandle, OpenVR.k_unTrackedDeviceIndex_Hmd, position, rotation);

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, 0.5f);
    ShowOverlay(overlayHandle);
} 

...

+ // Pass deviceIndex as argument.
+ private void SetOverlayTransformRelative(ulong handle, uint deviceIndex, Vector3 position, Quaternion rotation)
+ {
+     var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
+     var matrix = rigidTransform.ToHmdMatrix34();
+     var error = OpenVR.Overlay.SetOverlayTransformTrackedDeviceRelative(handle, deviceIndex, ref matrix);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to set overlay position: " + error);
+     }
+ }

Follow the controller

Overlay following the controller

Use controller device index instead of HMD to make the overlay follow a controller.

Get the controller device index

Get the left controller’s device index with GetTrackedDeviceIndexForControllerRole() of OpenVR.System.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var position = new Vector3(0, 0, 2);
    var rotation = Quaternion.Euler(0, 0, 0);
+   var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    SetOverlayTransformRelative(overlayHandle, OpenVR.k_unTrackedDeviceIndex_Hmd, position, rotation);

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, 0.5f);
    ShowOverlay(overlayHandle);
}

The argument is:
Left controller: EtrackedControllerRole.LeftHand
Right controller: EtrackedControllerRole.RightHand

If it fails to get the device index like the controller is disconnected, GetTrackedDeviceIndexForControllerRole() returns k_unTrackedDeviceIndexInvalid.

Follow the controller

We have got the left controller index, then make the overlay follow the controller. Pass the controller index to SetOverlayTransformRelative() that we previously created for the HMD.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var position = new Vector3(0, 0, 2);
    var rotation = Quaternion.Euler(0, 0, 0);
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
-   SetOverlayTransformRelative(overlayHandle, OpenVR.k_unTrackedDeviceIndex_Hmd, position, rotation);
+   if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
+   {
+       SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
+   }

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, 0.5f);

    ShowOverlay(overlayHandle);
}

Make sure the left controller is connected to the SteamVR window, then run the program.

The overlay should follow the controller instead of the HMD.

Adjust overlay position

To make a watch application, we will adjust the overlay position on the left wrist. Make position parameters editable at runtime on the Unity editor.

Add member variables

Add size, position, and rotation variables as class members. Use Range() attribute to show sliders on the inspector.

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

+   [Range(0, 0.5f)] public float size = 0.5f;
+   [Range(-0.2f, 0.2f)] public float x;
+   [Range(-0.2f, 0.2f)] public float y;
+   [Range(-0.2f, 0.2f)] public float z;
+   [Range(0, 360)] public int rotationX;
+   [Range(0, 360)] public int rotationY;
+   [Range(0, 360)] public int rotationZ;

    ...

Sliders are shown on the inspector.

Replace variables in the code

Replace the existing size and position variables in the code with the added member variables.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

-   var position = new Vector3(0, 0, 2);
-   var rotation = Quaternion.Euler(0, 0, 0);
+   var position = new Vector3(x, y, z);
+   var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ); 
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
    }

    SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

-   SetOverlaySize(overlayHandle, 0.5f);
+   SetOverlaySize(overlayHandle, size);
    ShowOverlay(overlayHandle);
}

Update size and position in Update()

Make the size and position editable at runtime by adding code to the Update(). Note that this code will be deleted later. It is temporary for determining new positions and rotation.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var position = new Vector3(x, y, z);
    var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
    var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
    if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
    {
        SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
    }

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, size);
    ShowOverlay(overlayHandle);
}

+ private void Update()
+ {
+     SetOverlaySize(overlayHandle, size);
+ 
+     var position = new Vector3(x, y, z);
+     var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ); 
+     var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
+     if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
+     {
+         SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
+     }
+ }

...

Run the program. Ensure the inspector slider changes the overlay size and position at runtime.

Adjust overlay position

Move sliders to adjust the overlay position to the left wrist. I recommend changing sliders from the desktop window that is opened in the SteamVR dashboard.

Control the Unity editor from the SteamVR dashboard.

Another way is looking at the HMD view on the desktop and adjusting parameters.

It’s helpful if you don’t want to put on the HMD.

Here are sample parameters.

size = 0.08
x = -0.044
y = 0.015
z = -0.131
rotationX = 154
rotationY = 262
rotationZ = 0

When adjusting is done, right click on the WatchOverlay component name, and select Copy Component.

Stop the program, right click the WatchOverlay component again, and paste the copied values with Paste Component Value.

Run the program. Check the overlay is on the left wrist.

Remove temporary code

Remove the code from the Update().

private void Update()
{
-   SetOverlaySize(overlayHandle, size);
-   
-   var position = new Vector3(x, y, z);
-   var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ); 
-   var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
-   if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
-   {
-       SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
-   }
}

When the controller is not connected

Currently, the controller must be connected at launch because we get the controller device index at the Start().
Move the getting device index code from Start() to Update() to support the cases where the controller connects or disconnects in the middle.

private void Start()
{
    InitOpenVR();
    OverlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

-   var position = new Vector3(x, y, z);
-   var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
-   var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
-   if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
-   {
-       SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
-   }

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    SetOverlaySize(overlayHandle, size);

    ShowOverlay(overlayHandle);
}

+ private void Update()
+ {
+     var position = new Vector3(x, y, z);
+     var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
+     var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
+     if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
+     {
+         SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
+     }
+ }

Now, it works even if the controller is connected in the middle.

Final code

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    [Range(0, 0.5f)] public float size;
    [Range(-0.2f, 0.2f)] public float x;
    [Range(-0.2f, 0.2f)] public float y;
    [Range(-0.2f, 0.2f)] public float z;
    [Range(0, 360)] public int rotationX;
    [Range(0, 360)] public int rotationY;
    [Range(0, 360)] public int rotationZ;

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);

        SetOverlaySize(overlayHandle, size);
        ShowOverlay(overlayHandle);
    }

    private void Update()
    {
        var position = new Vector3(x, y, z);
        var rotation = Quaternion.Euler(rotationX, rotationY, rotationZ);
        var leftControllerIndex = OpenVR.System.GetTrackedDeviceIndexForControllerRole(ETrackedControllerRole.LeftHand);
        if (leftControllerIndex != OpenVR.k_unTrackedDeviceIndexInvalid)
        {
            SetOverlayTransformRelative(overlayHandle, leftControllerIndex, position, rotation);
        }
    }

    private void OnApplicationQuit()
    {
        DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
        ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

    private void ShutdownOpenVR()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }

    private ulong CreateOverlay(string key, string name)
    {
        var handle = OpenVR.k_ulOverlayHandleInvalid;
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create overlay: " + error);
        }

        return handle;
    }

    private void DestroyOverlay(ulong handle)
    {
        if (handle != OpenVR.k_ulOverlayHandleInvalid)
        {
            var error = OpenVR.Overlay.DestroyOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to dispose overlay: " + error);
            }
        }
    }

    private void SetOverlayFromFile(ulong handle, string path)
    {
        var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to draw image file: " + error);
        }
    }

    private void ShowOverlay(ulong handle)
    {
        var error = OpenVR.Overlay.ShowOverlay(handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to show overlay: " + error);
        }
    }

    private void SetOverlaySize(ulong handle, float size)
    {
        var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay size: " + error);
        }
    }

    private void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
    {
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }
    }

    private void SetOverlayTransformRelative(ulong handle, uint deviceIndex, Vector3 position, Quaternion rotation)
    {
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        var error = OpenVR.Overlay.SetOverlayTransformTrackedDeviceRelative(handle, deviceIndex, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }
    }
}

Here, We attach the overlay on the left wrist. Let’s draw Unity camera output to the overlay instead of the static image file in the next part.

Next: Draw Camera Output

SteamVR Overlay with Unity: Change Size and Position

kurohuku — Sun, 16 Jun 2024 19:04:04 +0000

Change overlay size

Pass width SetOverlayWidthInMeters() to set an overlay size (read the wiki for details). The width unit is meters. Height is automatically calculated based on the image aspect ratio. The default width is 1 m.
Change the overlay width to 0.5 m.

private void Start()
{        
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

+   var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to set overlay size: " + error);
+   }

    ShowOverlay(overlayHandle);
}

Run the program, the overlay should be shown as half size.

Set overlay absolute position

Let’s display the overlay at the absolute position in the VR space with SetOverlayTransformAbsolute(). (read the wiki for details)

Prepare position and rotation

We will put the overlay at the absolute position that moved to 2 m in the Y-axis (upper direction), and 3 m in the Z-axis (forward direction). Also, rotate it 45 degrees around the Z-axis.

First, prepare the position and rotation with Vector3 and Quaternion.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to set overlay size: " + error);
    }

+   var position = new Vector3(0, 2, 3);
+   var rotation = Quaternion.Euler(0, 0, 45);

    ShowOverlay(overlayHandle);
}

Create transformation matrix

Overlay position is calculated by multiplying an origin and transformation matrix.
The origin is the base position like the center of the play area that is defined in ETrackingUniverseOrigin.
The transformation matrix is represented as HmdMatrix34_t type.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to set overlay size: " + error);
    }

    var position = new Vector3(0, 2, 3);
    var rotation = Quaternion.Euler(0, 0, 45);
+   var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
+   var matrix = rigidTransform.ToHmdMatrix34();

    ShowOverlay(overlayHandle);
}

Change overlay position

Pass the transformation matrix to SetOverlayTransformAbsolute() to change the overlay position.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

    var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to set overlay size: " + error);
    }

    var position = new Vector3(0, 2, 3);
    var rotation = Quaternion.Euler(0, 0, 45);
    var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
    var matrix = rigidTransform.ToHmdMatrix34();
+   error = OpenVR.Overlay.SetOverlayTransformAbsolute(overlayHandle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to set overlay position: " + error);
+   }

    ShowOverlay(overlayHandle);
    ...

Arguments of SetOverlayTransformAbsolute()

SetOverlayTransformAbsolute() has three arguments.

The 1st is the overlay handle.

The 2nd ETrackingUniverseOrigin.TrackingUniverseStanding means that the floor center of the play area is the origin of the transform.
Otherwise, ETrackingUniverseOrigin.TrackingUniverseSeated uses the position that the user has reset as the seated position.

The 3rd, ref matrix is a reference to the transformation matrix.

Check the program

Run the program. The overlay position should be changed.

Move upward to 2 m, forward to 3 m, and rotate 45 degrees in the Z-axis from the play area origin

Optional: Left-handed and Right-handed system

Unity uses a left-handed system, and OpenVR uses a right-handed system.
In OpenVR, +y is up, +x is right, and -z is forward.
For now, we don’t need to pay attention to the difference because the SteamVR_Utils.RigidTransform internally converts the two systems.
However, when you make the transformation matrix manually, be careful that the Z-axis and the rotation directions are reversed.

Organize code

Set size

Move the size setting code into SetOverlaySize().

public class WatchOverlay : MonoBehaviour
{
    ...

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);

-       var error = OpenVR.Overlay.SetOverlayWidthInMeters(overlayHandle, 0.5f);
-       if (error != EVROverlayError.None)
-       {
-           throw new Exception("Failed to set overlay size: " + error);
-       }
+       SetOverlaySize(overlayHandle, 0.5f);

        var position = new Vector3(0, 2, 3);
        var rotation = Quaternion.Euler(0, 0, 45);
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        error = OpenVR.Overlay.SetOverlayTransformAbsolute(overlayHandle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }

        ShowOverlay(overlayHandle);
    }

    ...

+   private void SetOverlaySize(ulong handle, float size)
+   {
+       var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
+       if (error != EVROverlayError.None)
+       {
+           throw new Exception("Failed to set overlay size: " + error);
+       }
+   }
}

Set position

Move the position setting into SetOverlayTransformAbsolute().

public class WatchOverlay : MonoBehaviour
{
    ...

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);

        SetOverlaySize(overlayHandle, 0.5f);

        var position = new Vector3(0, 2, 3);
        var rotation = Quaternion.Euler(0, 0, 45);
-       var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
-       var matrix = rigidTransform.ToHmdMatrix34();
-       error = OpenVR.Overlay.SetOverlayTransformAbsolute(overlayHandle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
-       if (error != EVROverlayError.None)
-       {
-           throw new Exception("Failed to set overlay position: " + error);
-       }
+       SetOverlayTransformAbsolute(overlayHandle, position, rotation);

        ShowOverlay(overlayHandle);
    }

    ...

+   private void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
+   {
+       var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
+       var matrix = rigidTransform.ToHmdMatrix34();
+       var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
+       if (error != EVROverlayError.None)
+       {
+           throw new Exception("Failed to set overlay position: " + error);
+       }
+   }
}

Final code

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {        
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);

        SetOverlaySize(overlayHandle, 0.5f);

        var position = new Vector3(0, 2, 3);
        var rotation = Quaternion.Euler(0, 0, 45);
        SetOverlayTransformAbsolute(overlayHandle, position, rotation);

        ShowOverlay(overlayHandle);
    }

    private void OnApplicationQuit()
    {
        DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
        ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

    private void ShutdownOpenVR()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }

    private ulong CreateOverlay(string key, string name)
    {
        var handle = OpenVR.k_ulOverlayHandleInvalid;
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
        if (error != EVROverlayError.None)
        { 
            throw new Exception("Failed to create overlay: " + error);
        }
        return handle;
    }

    private void DestroyOverlay(ulong handle)
    {
        if (handle != OpenVR.k_ulOverlayHandleInvalid)
        {
            var error = OpenVR.Overlay.DestroyOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to dispose overlay: " + error);
            }
        }
    }

    private void SetOverlayFromFile(ulong handle, string path)
    {
        var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to draw image file: " + error);
        }
    }

    private void ShowOverlay(ulong handle)
    {
        var error = OpenVR.Overlay.ShowOverlay(handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to show overlay: " + error);
        }
    }

    private void SetOverlaySize(ulong handle, float size)
    {
        var error = OpenVR.Overlay.SetOverlayWidthInMeters(handle, size);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay size: " + error);
        }
    }

    private void SetOverlayTransformAbsolute(ulong handle, Vector3 position, Quaternion rotation)
    {
        var rigidTransform = new SteamVR_Utils.RigidTransform(position, rotation);
        var matrix = rigidTransform.ToHmdMatrix34();
        var error = OpenVR.Overlay.SetOverlayTransformAbsolute(handle, ETrackingUniverseOrigin.TrackingUniverseStanding, ref matrix);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to set overlay position: " + error);
        }
    }
}

Here, we have set the overlay size and position. But we need to attach the overlay to controllers instead of the absolute position to create a watch application.
Next part, we will make the overlay follow devices such as HMD or controllers.

Next: Follow Devices

SteamVR Overlay with Unity: Draw Image

kurohuku — Sun, 16 Jun 2024 19:02:18 +0000

Prepare an image file

Prepare an image file that satisfies the following conditions.

Size is lower than 1920 x 1080 px.
File format is in PNG, JPG, or TGA (24 or 32 bit color)

From here on, I explain with my SNS icon named “sns-icon.jpg”.

Put the image file inside Assets/StreamingAssets folder.

Draw image

Call SetOverlayFromFile() to draw an image file. (read the wiki for details)

void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

+   var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
+   var error = OpenVR.Overlay.SetOverlayFromFile(overlayHandle, filePath);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to draw image file: " + error);
+   }
}

Pass the overlay handle and image file path to SetOverlayFromFile().
We use the StreamingAssets folder to get the file path at runtime with Application.streamingAssetsPath.
Error handling remains the same as before.

Run the program, then launch the Overlay Viewer.
Click WatchOverlayKey and check the image is drawing on the right preview area.

Switch overlay visibility

The overlay is hidden by default. It is shown on the Overlay Viewer but not in the VR space.
Switch the overlay visibility with ShowOverlay() and HideOverlay(). (read the wiki for details)

Let’s show the overlay with ShowOverlay() at the start of the application.

private void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    var error = OpenVR.Overlay.SetOverlayFromFile(overlayHandle, filePath);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to draw image file: " + error);
    }

+   error = OpenVR.Overlay.ShowOverlay(overlayHandle);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to show overlay: " + error);
+   }
}

Run the program, put on your HMD, and look down at the floor. The overlay should appear at the center of the floor. Since overlays can’t be seen from the back, if you can’t view the overlay, move to the opposite side.

It shows during a VR game (the screenshot is Legendary Tales)

Organize code

Draw image file

Move the drawing code into SetOverlayFromFile().

void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
-   var error = OpenVR.Overlay.SetOverlayFromFile(overlayHandle, filePath);
-   if (error != EVROverlayError.None)
-   {
-       throw new Exception("Failed to draw image file: " + error);
-   }
+   SetOverlayFromFile(overlayHandle, filePath);

    error = OpenVR.Overlay.ShowOverlay(overlayHandle);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to show overlay: " + error);
    }
}

...

+ // Variable names changed
+ // overlayHandle -> handle
+ // filePath -> path
+ private void SetOverlayFromFile(ulong handle, string path)
+ {
+     var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to draw image file: " + error);
+     }
+ }

Show overlay

Move the switching overlay visibility into ShowOverlay().

void Start()
{
    InitOpenVR();
    overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

    var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
    SetOverlayFromFile(overlayHandle, filePath);

-   error = OpenVR.Overlay.ShowOverlay(overlayHandle);
-   if (error != EVROverlayError.None)
-   {
-       throw new Exception("Failed to show overlay: " + error);
-   }
+   ShowOverlay(overlayHandle);
}

...

+ private void ShowOverlay(ulong handle)
+ {
+     var error = OpenVR.Overlay.ShowOverlay(handle);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to show overlay: " + error);
+     }
+ }
}

Final code

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");

        var filePath = Application.streamingAssetsPath + "/sns-icon.jpg";
        SetOverlayFromFile(overlayHandle, filePath);
        ShowOverlay(overlayHandle);
    }

    private void OnApplicationQuit()
    {
        DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
        ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

    private void ShutdownOpenVR()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }

    private ulong CreateOverlay(string key, string name)
    {
        var handle = OpenVR.k_ulOverlayHandleInvalid;
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create overlay: " + error);
        }
        return handle;
    }

    private void DestroyOverlay(ulong handle)
    {
        if (handle != OpenVR.k_ulOverlayHandleInvalid)
        {
            var error = OpenVR.Overlay.DestroyOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to dispose overlay: " + error);
            }
        }
    }

    private void SetOverlayFromFile(ulong handle, string path)
    {
        var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to draw image file: " + error);
        }
    }

    private void ShowOverlay(ulong handle)
    {
        var error = OpenVR.Overlay.ShowOverlay(handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to show overlay: " + error);
        }
    }
}

Here, we have displayed the image on the overlay. Next part, we will change the overlay size and position.

Next: Change Size and Position

SteamVR Overlay with Unity: Create Overlay

kurohuku — Sun, 16 Jun 2024 18:45:43 +0000

Key and Name

Overlay needs two strings key and name.
key is a unique string among all overlays.
name is just an overlay name sometimes shown to users.
We use "WatchOverlayKey" and "WatchOverlay".

void Start()
{
    InitOpenVR();

+   var key = "WatchOverlayKey";
+   var name = "WatchOverlay";
}

Overlay handle

Create a variable to save an overlay handle.
Similar to controlling a file with a file handle, overlays are controlled with an overlay handle.

void Start()
{
    InitOpenVR();

    var key = "WatchOverlayKey";
    var name = "WatchOverlay";
+   var overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
}

OpenVR.k_ulOverlayHandleInvalid means the overlay is not created. Overlay handle is ulong type.

Create an overlay and get a handle

Use CreateOverlay() to create an overlay. (read the wiki for details)
Pass the key and name and reference of overlayHandle to CreateOverlay().

void Start()
{
    InitOpenVR();

    var key = "WatchOverlayKey";
    var name = "WatchOverlay";
    var overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
+   var error = OpenVR.Overlay.CreateOverlay(key, name, ref overlayHandle);
}

CreateOverlay() sets overlay handle to the variable overlayHandle.
The return value is an overlay creation error. All overlay errors are defined in EVROverlayError.

Error handling

Add error handling for overlay creation.

void Start()
{
    InitOpenVR();

    var key = "WatchOverlayKey";
    var name = "WatchOverlay";
    var overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
    var error = OpenVR.Overlay.CreateOverlay(key, name, ref overlayHandle);
+   if (error != EVROverlayError.None)
+   {
+       throw new Exception("Failed to create overlay: " + error);
+   }
}

If there is no error, EVROverlayError.None is back.

Cleanup overlay

Add code to dispose of the overlay at the end of the application.

Move overlay handle

Move overlayHandle from Start() to the class member variable.

public class WatchOverlay : MonoBehaviour
{
+   private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        InitOpenVR();

        var key = "WatchOverlayKey";
        var name = "WatchOverlay";
-       var overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref overlayHandle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create overlay: " + error);
        }
    }

    ...
}

Dispose of overlay

The function for overlay cleanup is DestroyOverlay(). (read the wiki for details)
Add OnApplicationQut() with overlay cleanup code.

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        InitOpenVR();

        var key = "WatchOverlayKey";
        var name = "WatchOverlay";
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref overlayHandle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create overlay: " + error);
        }
    }

+   private void OnApplicationQuit()
+   {
+       if (overlayHandle != OpenVR.k_ulOverlayHandleInvalid)
+       {
+           var error = OpenVR.Overlay.DestroyOverlay(overlayHandle);
+           if (error != EVROverlayError.None)
+           {
+               throw new Exception("Failed to dispose overlay: " + error);
+           }
+       }
+   }

    private void OnDestroy()
    {
        ShutdownOpenVR();
    }

    ...
}

DestroyOverlay() must be called before Shutdown() so we put the overlay cleanup code inside OnApplicationQuit() that is called before OnDestroy().

Check created overlay

Check whether the overlay is created.
Overlay Viewer included in the SteamVR is useful to check overlays.

Select Developer > Overlay Viewer in the SteamVR menu.

Overlay Viewer displays a list of created overlays. You can see many overlays already created by SteamVR system.

In this state, run the program from Unity.

Then the overlay key WatchOverlayKey is added to the list. You can see the overlay details by clicking the key.

The overlay preview is shown on the right side gray area but now we draw nothing to the overlay yet.

Here, we confirmed the overlay is created. Stop the program and close Overlay Viewer.

Optional: Overlay Viewer location

Overlay Viewer is located in the SteamVR install directory.
C:/ProgramFiles(x86)/Steam/steamapps/common/SteamVR/bin/win32/overlay_viewer.exe
We use this tool frequently during development so I recommend creating short cut.

Organize code

Create overlay

Move the overlay creation code intoCreateOverlay(). It gets key and name as arguments, and return created overlay handle. Please note that some variable names are changed in code organization.

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        InitOpenVR();

-       var key = "WatchOverlayKey";
-       var name = "WatchOverlay";
-       var error = OpenVR.Overlay.CreateOverlay(key, name, ref overlayHandle);
-       if (error != EVROverlayError.None)
-       {
-           throw new Exception("Failed to create overlay: " + error);
-       }
+       overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");
    }

    ...

+   private ulong CreateOverlay(string key, string name) {
+       // Some code changed here.
+       var handle = OpenVR.k_ulOverlayHandleInvalid;
+       var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
+       if (error != EVROverlayError.None)
+       {
+           throw new Exception("Failed to create overlay: " + error);
+       }
+       return handle;
+   }
}

Cleanup overlay

Similarly, move the overlay cleanup code into DestroyOverlay().

public class WatchOverlay : MonoBehaviour
{
    ...

    private void OnApplicationQuit()
    {
-       if (overlayHandle != OpenVR.k_ulOverlayHandleInvalid)
-       {
-           var error = OpenVR.Overlay.DestroyOverlay(overlayHandle);
-           if (error != EVROverlayError.None)
-           {
-               throw new Exception("Failed to dispose overlay: " + error);
-           }
-       }
+       DestroyOverlay(overlayHandle);
    }

    ...

+   // overlayHandle -> handle variable name changed.
+   private void DestroyOverlay(ulong handle)
+   {
+       if (handle != OpenVR.k_ulOverlayHandleInvalid)
+       {
+           var error = OpenVR.Overlay.DestroyOverlay(handle);
+           if (error != EVROverlayError.None)
+           {
+               throw new Exception("Failed to dispose overlay: " + error);
+           }
+       }
+   }
}

Final code

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;

    private void Start()
    {
        InitOpenVR();
        overlayHandle = CreateOverlay("WatchOverlayKey", "WatchOverlay");
    }

    private void OnApplicationQuit()
    {
        DestroyOverlay(overlayHandle);
    }

    private void OnDestroy()
    {
        ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

    private void ShutdownOpenVR()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }

    private ulong CreateOverlay(string key, string name)
    {
        var handle = OpenVR.k_ulOverlayHandleInvalid;
        var error = OpenVR.Overlay.CreateOverlay(key, name, ref handle);
        if (error != EVROverlayError.None)
        {
            throw new Exception("Failed to create overlay: " + error);
        }
        return handle;
    }

    private void DestroyOverlay(ulong handle)
    {
        if (handle != OpenVR.k_ulOverlayHandleInvalid)
        {
            var error = OpenVR.Overlay.DestroyOverlay(handle);
            if (error != EVROverlayError.None)
            {
                throw new Exception("Failed to dispose overlay.: " + error);
            }
        }
    }
}

With that, we have created and cleaned up an overlay. Next part, we will draw an image file to the overlay.

Next: Draw Image

SteamVR Overlay with Unity: Initialize OpenVR

kurohuku — Sun, 16 Jun 2024 10:57:44 +0000

Create a new script

Create a folder named Scripts inside Assets. Then create a file named WatchOverlay.cs inside the Scripts folder. We will add our overlay code to this script.

Copy the following code to WatchOverlay.cs.

using UnityEngine;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
    }
}

Put the script to the scene

In the hierarchy window, right click > Create Empty.
Change the object name to WatchOverlay, then drag the WatchOverlay.cs file from the project window to the newly created object.

Add using directive

Add using Valve.VR to WatchOverlay.cs to use the OpenVR API. This namespace is included in the SteamVR Plugin.

using UnityEngine;
+ using Valve.VR;

public class WatchOverlay : MonoBehaviour
{
   private void Start()
   {
   }
}

Initialize OpenVR

Initialize OpenVR with Init() to use the API. (read the wiki for details)

using UnityEngine;
using Valve.VR;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
+       var error = EVRInitError.None;
+       OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
    }
}

We pass two arguments to Init().

The first argument is a reference. If an error occurs during initialization, Init() sets the variable's value referenced to the error. All errors are defined in EVRInitError.

The second argument is the application type, which will be EVRApplicationType.VRApplication_Overlay for overlay applications. All application types are defined in EVRApplicationType.

If the application has been initialized as an overlay, it can be run with other VR applications at the same time.

Initialize error handling

Add error handling.

using UnityEngine;
using Valve.VR;
+ using System;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
+       if (error != EVRInitError.None)
+       {
+           throw new Exception("Failed to initialize OpenVR: " + error);
+       }
    }
}

If the initialization succeeds, the error value will remain EVRInitError.None. We will throw an Exception if this is not the case. Note that you need to add using System to use Exception.

If OpenVR is already initialized

If OpenVR has already been initialized, skip the initialization.

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
+       if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }
}

If OpenVR has not been initialized, OpenVR.System is null.
The API is broken down into parts such as the following. They are null before initialization.

OpenVR.System
OpenVR.Chaperone
OpenVR.Compositor
OpenVR.Overlay
OpenVR.RenderModels
OpenVR.Screenshots
OpenVR.Input

Cleanup

Add cleanup to dispose of the initialized OpenVR.
Call Shutdown() at the end of the program. (read the wiki for details)

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

+   private void OnDestroy()
+   {
+       if (OpenVR.System != null)
+       {
+           OpenVR.Shutdown();
+       }
+   }
}

Check the initialization works

Let's access the initialized OpenVR API.

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
        if (OpenVR.System != null) return;

+       Debug.Log(OpenVR.Overlay);

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
            return;
        }

+       Debug.Log(OpenVR.Overlay);
    }

    private void OnDestroy()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }
}

We can see that the API is available after initialization.
If the SteamVR has not launched, it should be automatically launched by Init().

Optional: Initialize OpenVR by SteamVR Plugin setting

We don’t use it in this tutorial, but the SteamVR Unity Plugin has an automatic initialization setting.
Check Initialize XR on Project Settings > Startup In XR Plug-in Management.

Select Overlay as the Application Type in the OpenVR setting.

Delete the Debug.Log().

private void Start()
{
    if (OpenVR.System != null) return;

-   Debug.Log(OpenVR.Overlay);
    var error = EVRInitError.None;
    OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
    if (error != EVRInitError.None)
    {
        throw new Exception("Failed to initialize OpenVR: " + error);
        return;
    }
-   Debug.Log(OpenVR.Overlay);
}

Organize code

We have initialized and cleaned up OpenVR. Let’s organize the code here.
Sometimes this tutorial will organize code to keep methods small and clean.

Initialize

Move the initialization code into InitOpenVR().

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
+       InitOpenVR();
-       if (OpenVR.System != null) return;
-
-       var error = EVRInitError.None;
-       OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
-       if (error != EVRInitError.None)
-       {
-           throw new Exception("Failed to initialize OpenVR: " + error);
-       }
    }

    private void OnDestroy()
    {
        if (OpenVR.System != null)
        {
            OpenVR.Shutdown();
        }
    }

+   private void InitOpenVR()
+   {
+       if (OpenVR.System != null) return;
+
+       var error = EVRInitError.None;
+       OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
+       if (error != EVRInitError.None)
+       {
+           throw new Exception("Failed to initialize OpenVR: " + error);
+       }
+   }
}

Optional: If you want to use `return` instead of `throw`

This tutorial uses throw for readability of error handling. If you prefer return the error code or a boolean, replace the error handling code with the following.

// Return error code
private bool InitOpenVR()
{
    if (OpenVR.System != null) return;

    var error = EVRInitError.None;
    OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
    if (error != EVRInitError.None)
    {
        Debug.LogError("Failed to initialize OpenVR: " + error);
    }
    return error;
}

// Return bool value
private bool InitOpenVR()
{
    if (OpenVR.System != null) return;

    var error = EVRInitError.None;
    OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
    if (error != EVRInitError.None)
    {
        Debug.LogError("Failed to initialize OpenVR: " + error);
        return false;
    }
    return true;
}

Cleanup

Similarly, move the cleanup code to ShutdownOpenVR().

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
       InitOpenVR();
    }

    private void OnDestroy()
    {
-       if (OpenVR.System != null)
-       {
-           OpenVR.Shutdown();
-       }
+       ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

+   private void ShutdownOpenVR()
+   {
+       if (OpenVR.System != null)
+       {
+           OpenVR.Shutdown();
+       }
+   }
}

Final code

using UnityEngine;
using Valve.VR;
using System;

public class WatchOverlay : MonoBehaviour
{
    private void Start()
    {
       InitOpenVR();
    }

    private void OnDestroy()
    {
       ShutdownOpenVR();
    }

    private void InitOpenVR()
    {
        if (OpenVR.System != null) return;

        var error = EVRInitError.None;
        OpenVR.Init(ref error, EVRApplicationType.VRApplication_Overlay);
        if (error != EVRInitError.None)
        {
            throw new Exception("Failed to initialize OpenVR: " + error);
        }
    }

   private void ShutdownOpenVR()
   {
       if (OpenVR.System != null)
       {
           OpenVR.Shutdown();
       }
   }
}

🎉 We have initialized and cleaned up OpenVR. Now we can access OpenVR APIs. In the next part, we will create an overlay with this API.

Next: Create Overlay

DEV Community: kurohuku

Get controller device index without GetTrackedDeviceIndexForControllerRole()

Overview

Deprecated Functions

Create Unity Project

Instlal SteamVR Plugin

Controller Binding

If the "Poses" button is not shown

Script Creation

Execution

SteamVR Overlay with Unity: Appendix

OpenVR repository

SteamVR Unity Plugin repository

Other repositories

Show 3d objects with overlay

Method 1: SideBySide

Method 2: Stereo Panorama

Method 3: Overlay projection

Method 4: Combine planes

Get the HMD output

Get controller device index without deprecated function GetTrackedDeviceIndexForControllerRole

SteamVR Overlay with Unity: Controller Input

Prerequisite setting

Create action manifest

Generate action manifest

Create default binding

How about other controller bindings?

Create script

Initialize and cleanup OpenVR

Set action manifest path

Get action set handle

Get action handle

Update action state

Prepare action set to update

Update state

Get action value

GetDigitalActionData()

GetAnalogActionData()

GetPoseActionDataRelativeToNow()

GetPoseActionDataForNextFrame()

Create event handler

Notify to Unity Event

Attach event handler

Add code to show/hide the watch

Hide watch by default

Show watch by WakeUp action

Add hide method

Hide watch after three seconds

Complete 🎉

Part 10 — Controller Input

Prerequisite setting

Create action manifest

Generate action manifest

Create default binding

How about other controller bindings?

Create script

Initialize and cleanup OpenVR

Set action manifest path

Get action set handle

Get action handle

Update action state

Prepare action set to update

Get action value

GetDigitalActionData()

GetAnalogActionData()

GetPoseActionData()

Create event handler

Notify to Unity Event

Attach event handler

Add code to show/hide the watch

Hide watch by default

Show watch by WakeUp action

Add hide method

Hide watch after three seconds

Complete 🎉

SteamVR Overlay with Unity: Overlay Events

Show the current time on the right hand

Create hand variable

Get selected controller device index

Note down current params

Get controller device index without deprecated function `GetTrackedDeviceIndexForControllerRole`