Multi-Directional Navigation

tl;dr demo, repo

In my previous job, I had the opportunity to work on a web application for a video game company. This web app is embedded within AAA games on PlayStation 4, Xbox One and Nintendo Switch. I want to share what I've learned during this experience, especially how to manage the navigation.

UI for a TV Screen 📺

Most web developers are now used to develop responsive user interfaces for mobile, tablet and desktop computers. Your webpage should provide a user experience for people using a touch screen, a mouse, a keyboard, a screen reader...
In our case however, the app gets rendered on television screens! or on the Switch screen when being used in portable mode.
Gaming systems, even those supporting 4K resolutions, will render our web app in a 1080p resolution (1920x1080 pixels viewport). Other might render it in 720p (1280x720 pixels viewport). Each has their specificity, for instance, the Nintendo Switch reserves an area at the bottom of the screen to display their own footer.
To handle all those resolutions, we better work on an adaptive design. The content (such as the system logo) and its size will adapt to each system and its resolution. There is no reason to worry about unsupported resolutions here, simply because the user can't resize their viewport.
People use a gamepad to navigate in the app. The goal is to provide them an User Experience that is similar to the one they see in-game. So we don't want to display a mouse cursor or scroll bars, this might break their momentum and create frustration.

Here's a list of tips:

• Display a legend somewhere to indicate which button can be pressed and what action do they trigger. For example, you want to tell them "by pressing this button you will go back".
• Look at existing game menus and dashboards. You want to use all the space available in the viewport and have some "fixed" content (e.g. a menu, header, footer...). Viewport units, REM and CSS Grid help a lot! Some browsers might not support all those cool features, you can fallback to something else like flexbox.
• Highlight which element is focused. If you use React in your project you might want to try styled-components. It lets you create components that have a dynamic style based on their props in a very smooth way.
• The URL is your friend. You can tell the gaming system to hide it. So the user won't be able to modify its content. Use it to do conditional rendering and to pass information from the game to your app via query strings.
• You can also use Node environment variables to create different builds to support different systems.
• Not all your teammates have a dev kit to start a game and test your app. Deploying a private version usable from any computer via its keyboard and tools such as Storybook helps a lot.

Gamepad Navigation 🎮

The UI is made of focusable elements where the user can navigate in at least four directions: up, down, left and right.

Browsers don't support such navigation natively. You might have heard about web accessibility that lets you use tab and shift+tab to focus elements one by one. Accessibility best practices are a good source of inspiration. You might wonder, why not using the gamepad api? Fun fact, not all gaming system browsers support it. We instead ask the system to map each button as keyboard keys. The goal is to create a custom focus manager that will take care of:

• inserting/updating/removing elements in a list;
• programmatically focusing an element based on a direction.

In my demo which uses React, I opted for useReducer and the Context API. But the logic can be reused with any other state management solution, such as Redux. I won't go into the implementation details, here are the main takeaways:

• Each focusable element is represented by an object containing a unique id, its position (top, bottom, left, right) and its depth. We can use Element.getBoundingClientRect() or pass our own custom values.
• Imagine the depth like an equivalent of the z-index in CSS. It let us handle different layers of focusable elements, such as a modal.
• We use a global event listener to listen to keyboard inputs. When matching one of the arrow keys we find the closest neighbour based on the current focused element and the current depth. My function to find the closest neighbour can be overriden. We could imagine different algorithms to find the next focused element depending on the current page.
• Then it's up to you to create custom hooks and to have fun! E.g. in my app I am playing a "move" sound effect when the current focus id changes. Check this article if you want to useSound too!

Testing 🤖

Automated tests and continuous integration improve your confidence when shipping code.
It's very important to write unit tests for vital parts of your apps, like the pure functions that are used to find the closest neighbour. I like writing snapshot tests for my Styled Components which have dynamic styles. I also have a few integration tests made with React Testing Library.
But I believe that end-to-end tests are the best here because they are very natural to write and will cover all your business logic without needing to mock everything. That's why most of my hooks have no dedicated tests. For example here is a test made with Cypress that visits a page, opens a modal, navigates within it and then closes it.

Thanks for reading, let me know if you have questions!

• Check the demo;
• Browse the repo.

Merci Jean-Loup for proofreading. He was also one of my teammates!

Comments

[Sylvain Hamann]

New features added:

• set custom keys via query parameters
• use a gamepad
• use your voice