DEV Community

kurohuku
kurohuku

Posted on • Updated on

SteamVR Overlay with Unity: Controller Input

In this part, we will make the watch hidden by default and show it in a few seconds by pressing a controller button.

Image description

There are various ways to get controller input with Unity. This time, we will use OpenVR Input API (SteamVR Input).

Prerequisite setting

Open the SteamVR Settings.

Image description

Toggle Advanced Settings.

Image description

Toggle Developer > Enable debugging options in the input binding user interface to On.

Image description

Create action manifest

SteamVR Input uses abstracted actions instead of direct access to the physical buttons.
The developer predefines actions and may create default mappings for each controller. Users can assign physical buttons to the actions in the SteamVR Input setting window.

Image description

First, create the action manifest that defines actions in JSON format. We use the Unity SteamVR Plugin GUI tool to generate the action manifest easily.

Generate action manifest

Select Unity menu Window > SteamVR Input.

Image description

It asks you whether you want to use a sample action manifest file.
This time we will make up from the first so select No.

Image description

Input the action sets name to “Watch”, and select the below dropdown to per hand. An application can have multiple action sets.

Image description

Per hand means it can map different actions to the buttons of the left and right-hand controller separately.
Mirrored means it can map actions to only one controller and the other buttons are mirrored to the mapping.

Click the NewAction name that is in the In box of the Actions section.
The “Action Details” is shown to the right. Change the Name to “WakeUp”.
Click the “Save and generate” button at the left bottom to generate an action manifest file.

Image description

It will generate a new action manifest file as StreamingAssets/SteamVR/actions.json.

{
  "actions": [
    {
      "name": "/actions/Watch/in/WakeUp",
      "type": "boolean"
    }
  ],
  "action_sets": [
    {
      "name": "/actions/Watch",
      "usage": "leftright"
    }
  ],
  "default_bindings": [],
  "localization": []
}
Enter fullscreen mode Exit fullscreen mode

Create default binding

Click the right bottom Open binding UI button.

Image description

If any VR controller is not shown, check your HMD is recognized from SteamVR.

Click the “Create New Binding”.

Image description

Set the Y button to show the watch for a few seconds. If your controller doesn’t have the Y button, use another button instead.

Click the + button to the right of the Y Button.

Image description

Select BUTTON.

Image description

Select None to the right of the Click, then select wakeup action.

Image description

Image description

Click the left bottom check icon to save changes.

Image description

Click the right bottom Replace Default Binding.

Image description

Click Save.

Image description

At this point, a new default binding file is added to the action manifest.

actions.json

{
   "action_sets" : [
      {
         "name" : "/actions/Watch",
         "usage" : "single"
      }
   ],
   "actions" : [
      {
         "name" : "/actions/Watch/in/WakeUp",
         "type" : "boolean"
      }
   ],
   "default_bindings" : [
      {
+        "binding_url" : "application_generated_unity_retryoverlay_exe_binding_oculus_touch.json",
+        "controller_type" : "oculus_touch"
      }
   ],
   "localization" : []
}
Enter fullscreen mode Exit fullscreen mode

The default binding file is generated into the same directory to actions.json. These files will be included in the build for users.

Image description

Click the <- BACK button at the upper left, then you see the new setting is active.

Image description

Now we’ve made the default binding. Close the SteamVR Input window.

Also, close the Unity SteamVR Input window.
At this time, it asks you to save the change, so click Close.

Image description

In this tutorial environment, clicking the Save button overwrites the action manifest as an empty default_bindings.
(There is no problem with clicking Save when we open the binding setting window next time.)


How about other controller bindings?

You can create other controller’s default bindings by clicking the controller name on the right side.

Image description

Other controllers’ default bindings are added to default_bindings param of action manifest.

{
    "action_sets" : [
       {
          "name" : "/actions/NewSet",
          "usage" : "leftright"
       }
    ],
    "actions" : [
       {
          "name" : "/actions/NewSet/in/NewAction",
          "type" : "boolean"
       }
    ],
    "default_bindings" : [
      {
         "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_oculus_touch.json",
         "controller_type" : "oculus_touch"
      },
+     {
+        "binding_url" : "application_generated_unity_steamvr_inputbindingtest_exe_binding_vive_controller.json",
+        "controller_type" : "vive_controller"
+     }
   ],
   "localization" : []
}
Enter fullscreen mode Exit fullscreen mode

You don’t necessarily have to create other controller default bindings because SteamVR automatically remaps actions for other controllers if at least one default binding exists.

Image description


Create script

Create a new script InputController.cs into Scripts folder.
We will add the controller input related code to this file.

On hierarchy, right click > Create Empty to create an empty object, change the object name to InputController.
Drag InputController.cs from the project window to the InputController object.

Image description

Initialize and cleanup OpenVR

Copy the below code to InputController.cs.

using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
Enter fullscreen mode Exit fullscreen mode

Set action manifest path

Set the action manifest path with SetActionManifestPath() at launch. (read the wiki for details)
The action manifest is generated as StreamingAssets/SteamVR/actions.json so we set this.

using System;
using UnityEngine;
using Valve.VR; 

public class InputController : MonoBehaviour
{
    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

+       var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to set action manifest path: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
Enter fullscreen mode Exit fullscreen mode

Get action set handle

An application can have some action sets that have multiple actions. We use an action set handle to determine which action set is used.
Get action set handle with GetActionSetHandle().

The action set is set with a string /actions/[action_set_name].
This time, it’s /actions/Watch.

public class InputController : MonoBehaviour
{
+   ulong actionSetHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to set action manifest path: " + error);
        }

+       error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Failed to get action set /actions/Watch: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
Enter fullscreen mode Exit fullscreen mode

Get action handle

Next, get the action handle with GetActionHandle().
Action is set with /actions/[action_set_name]/in/[action_name].
This time, action is /actions/Watch/in/Wakeup.

public class InputController : MonoBehaviour
{
    ulong actionSetHandle = 0;
+   ulong actionHandle = 0;

    private void Start()
    {
        OpenVRUtil.System.InitOpenVR();

        var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
        if (error != EVRInputError.None)
        {
            throw new Exception("Faild to set action manifest path: " + error);
        }

        error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get action set /actions/Watch: " + error);
        }

+       error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
+       if (error != EVRInputError.None)
+       {
+           throw new Exception("Faild to get action /actions/Watch/in/WakeUp: " + error);
+       }
    }

    private void Destroy()
    {
        OpenVRUtil.System.ShutdownOpenVR();
    }
}
Enter fullscreen mode Exit fullscreen mode

Update action state

Update each action state at the first of each frame.

Prepare action set to update

We use an action set to specify which actions will be updated.
Pass target action sets as an array of VRActiveActionSet_t type.
This time, the action set is only /actions/Watch so we make a single-element array VRActionActiveSet_t[].

private void Start()
{
    OpenVRUtil.System.InitOpenVR();

    var error = OpenVR.Input.SetActionManifestPath(Application.streamingAssetsPath + "/SteamVR/actions.json");
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to set action manifest path: " + error);
    }

    error = OpenVR.Input.GetActionSetHandle("/actions/Watch", ref actionSetHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action set /actions/Watch: " + error);
    }

    error = OpenVR.Input.GetActionHandle($"/actions/Watch/in/WakeUp", ref actionHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get action /actions/Watch/in/WakeUp: " + error);
    }
}

+ private void Update()
+ {
+     // Action set list to update
+     var actionSetList = new VRActiveActionSet_t[]
+     {
+         // This time, Watch action only.
+         new VRActiveActionSet_t()
+         {
+             // Pass Watch action set handle
+             ulActionSet =  actionSetHandle,
+             ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
+         }
+     };
+ }

...
Enter fullscreen mode Exit fullscreen mode

ulRestrictedToDevice limits input only from a specific device. For exmaple, to bind different actions for left and right controllers. We don't use this basically so set k_ulInvalidInputValueHandle.

Update state

We use UpdateActionState() to update action state. Pass the action set array to UpdateActionState().

private void Update()
{ 
   var actionSetList = new VRActiveActionSet_t[]
   {
       new VRActiveActionSet_t()
       {
           ulActionSet =  actionSetHandle,
           ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
       }
   };

+   var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));    
+   var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
+   if (error != EVRInputError.None)
+   {
+       throw new Exception("Faild to update action state: " + error);
+   }
}
Enter fullscreen mode Exit fullscreen mode

The second argument activeActionSize is the byte size of VRActiveActionSet_t the struct.

Get action value

We have updated the action state. Next, we will get the action value.
The function used to retrieve action value varies depending on the action type.

GetDigitalActionData()

This gets an on/off value likes the button is pushed or not.

GetAnalogActionData()

This gets analog values like thumb stick direction or how much trigger is pulled.

GetPoseActionData()

This gets a pose like controller position and rotation.

This time, we want to get the button on/off value so use GetDigitalActionData().
Get the action value with WakeUp action handle we prepared.

private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };

    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }

+   var result = new InputDigitalActionData_t();
+   var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
+   error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
+   if (error != EVRInputError.None)
+   {
+       throw new Exception("Failed to get WakeUp action data: " + error);
+   }
}
Enter fullscreen mode Exit fullscreen mode

The 3rd argument digitalActionSize is the byte size of InputDigitalActionData_t structure.
The 4th argument ulRestrictToDevice is generally not used so set OpenVR.k_ulInvalidInputValueHandle.

The returned value type is InputDigitalActionData_t.

struct InputDigitalActionData_t
{
    bool bActive;
    VRInputValueHandle_t activeOrigin;
    bool bState;
    bool bChanged;
    float fUpdateTime;
};
Enter fullscreen mode Exit fullscreen mode

The action on/off state is set to bState.
bChanged is true at the frame where the action state is changed.

Let’s detect the action.

private void Update()
{
    var actionSetList = new VRActiveActionSet_t[]
    {
        new VRActiveActionSet_t()
        {
            ulActionSet =  actionSetHandle,
            ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
        }
    };

    var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
    var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to update action state: " + error);
    }

    var result = new InputDigitalActionData_t();
    var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
    error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
    if (error != EVRInputError.None)
    {
        throw new Exception("Failed to get WakeUp action data: " + error);
    }

+   if (result.bState && result.bChanged)
+   {
+       Debug.Log("WakeUp is executed");
+   }
}
Enter fullscreen mode Exit fullscreen mode

Run the program and push the Y button, then the message is logged to the Unity console.

Create event handler

Notify to Unity Event

Add UnityEvent into InputController.cs to notify WakeUp action, then call it with Invoke().

using System;
using UnityEngine;
using Valve.VR;
+ using UnityEngine.Events;

public class InputController : MonoBehaviour
{
+   public UnityEvent OnWakeUp; 

    private ulong actionSetHandle = 0;
    private ulong actionHandle = 0;

    // code omit.

    private void Update()
    {
        var actionSetList = new VRActiveActionSet_t[]
        {
            new VRActiveActionSet_t()
            {
                ulActionSet =  actionSetHandle,
                ulRestrictedToDevice = OpenVR.k_ulInvalidInputValueHandle,
            }
        };

        var activeActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(VRActiveActionSet_t));
        var error = OpenVR.Input.UpdateActionState(actionSetList, activeActionSize);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to update action state: " + error);
        }

        var result = new InputDigitalActionData_t();
        var digitalActionSize = (uint)System.Runtime.InteropServices.Marshal.SizeOf(typeof(InputDigitalActionData_t));
        error = OpenVR.Input.GetDigitalActionData(actionHandle, ref result, digitalActionSize, OpenVR.k_ulInvalidInputValueHandle);
        if (error != EVRInputError.None)
        {
            throw new Exception("Failed to get WakeUp action data: " + error);
        }

        if (result.bState && result.bChanged)
        {
-           Debug.Log("WakeUp is executed");
+           OnWakeUp.Invoke();
        }
    }
Enter fullscreen mode Exit fullscreen mode

Attach event handler

Add method into WatchOverlay.cs that will be called when WakeUp action is executed.

public class WatchOverlay : MonoBehaviour
{

    ...

+   public void OnWakeUp()
+   {
+       // Show watch.
+   }
}
Enter fullscreen mode Exit fullscreen mode

In the hierarchy, open InputController inspector, then set WatchOverlay.OnWakeUp() to OnWakeUp() field.

Image description

Add code to show/hide the watch

Update WatchOverlay.cs to show the watch only when WakeUp action is executed.

Hide watch by default

private void Start()
{
    OpenVRUtil.System.InitOpenVR();
    overlayHandle = Overlay.CreateOverlay("WatchOverlayKey", "WatchOverlay");

    Overlay.FlipOverlayVertical(overlayHandle);
    Overlay.SetOverlaySize(overlayHandle, size);
-   Overlay.ShowOverlay(overlayHandle);
}
Enter fullscreen mode Exit fullscreen mode

Show watch by WakeUp action

Show overlay when the WakeUp action is executed.

public void OnWakeUp()
{
+   Overlay.ShowOverlay(overlayHandle);
}
Enter fullscreen mode Exit fullscreen mode

Add hide method

Add HideOverlay() to OpenVRUtil.cs.

public static void ShowOverlay(ulong handle)
{
    var error = OpenVR.Overlay.ShowOverlay(handle);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to show overlay: " + error);
    }
}

+ public static void HideOverlay(ulong handle)
+ {
+     var error = OpenVR.Overlay.HideOverlay(handle);
+     if (error != EVROverlayError.None)
+     {
+         throw new Exception("Failed to hide overlay: " + error);
+     }
+ }

public static void SetOverlayFromFile(ulong handle, string path)
{
    var error = OpenVR.Overlay.SetOverlayFromFile(handle, path);
    if (error != EVROverlayError.None)
    {
        throw new Exception("Failed to draw image file: " + error);
    }
}
Enter fullscreen mode Exit fullscreen mode

Hide watch after three seconds

This time, we use Unity Coroutine to wait three seconds.

WatchOverlay.cs

using System;
+ using System.Collections;
using UnityEngine;
using Valve.VR;
using OpenVRUtil;

public class WatchOverlay : MonoBehaviour
{
    public Camera camera;
    public RenderTexture renderTexture;
    public ETrackedControllerRole targetHand = ETrackedControllerRole.RightHand;
    private ulong overlayHandle = OpenVR.k_ulOverlayHandleInvalid;
+   private Coroutine sleepCoroutine;

    ...

    public void OnWakeUp()
    {
        Overlay.ShowOverlay(overlayHandle);
+       if (sleepCoroutine != null)
+       {
+           StopCoroutine(sleepCoroutine);
+       }
+       sleepCoroutine = StartCoroutine(Sleep());
    }

+   private IEnumerator Sleep()
+   {
+       yield return new WaitForSeconds(3);
+       Overlay.HideOverlay(overlayHandle);
+   }
}
Enter fullscreen mode Exit fullscreen mode

Run the program, and check the watch is shown for three seconds when the Y button is pushed.

Image description

Complete 🎉

The watch application is completed.
This tutorial ends here, so the final code organization is up to you.

Look back at what we touched on in this tutorial.

  • Initialize OpenVR
  • Create overlay
  • Draw image file
  • Change overlay size and position
  • Follow devices
  • Draw camera output
  • Create dashboard overlay
  • Process events
  • Controller input

We learned the basics of overlay application development.
OpenVR has many more APIs, so how about trying to make original tools?

The next page is additional information for more details.

Top comments (3)

Collapse
 
reavoend profile image
Reavo End

Thank you for the tutorial! This is a wonderful starting point for anyone who wants to get into VR overlay development.

There were a few little issues and parts where you skipped steps, but I'll just point out the major ones on this page:

  1. "Update action state" is missing several lines of code that exist in the next step, but are not highlighted in green
  2. "Create event handler" step has two of the "using" keyword

Thanks again for your great contributions to the VR community.

Collapse
 
kurohuku profile image
kurohuku

Thank you for reading! I fixed it!

Collapse
 
tx profile image
Dsel

The best Unity/VR tutorial I've ever seen. Very well done!