DEV Community: Florent Lagrede

How to reproduce Death Stranding UI with react and react-three-fiber

Florent Lagrede — Thu, 15 Apr 2021 17:15:13 +0000

In this demo, we will try to reproduce one of the main interface of the Death Stranding game.

About the game

Death Stranding is a game produced by Hideo Kojima (especially known for its Metal Gear series games). The game takes place in a post-apocalyptic future where an unknown phenomenon has ravaged most of the world. You play a character, Sam, responsible for making merchandise deliveries to the scattered remains of the population in a world that became quite dangerous. If Sam looks familiar to you it’s because its model is based on the actor who played Daryl in Walking Dead.

About this interface

On this interface, the player must arrange the merchandise he will carry from point A to point B.
The arrangement done by the player will have a significant consequence on the success of the delivery.

This interface is really interesting for a number of reasons:

The player is expected to spend some time in this interface so it is really important that it doesn’t break the flow of the game.
It should also keep the player fully immersed in the universe of the game
How it uses both a 2D overlay on top of a 3D Scene
Its aesthetic choices

For the sake of this article, I reduced the scope of the interface but I tried to keep the essence of what makes it interesting. Our goal will be to reproduce:

The 3D scene to display the merchandises
The 2D overlay to manage the merchandises
Keeping some interactions between the 2D overlay and the 3D scene

For the 3D scene, there will be 3 different positions to display the merchandises:

Private locker (the main storage)
Shared locker (alternative storage)
Sam cargo (represents merchandises carried by Sam)

Target audience

This article requires some knowledge about threejs and react-three-fiber.
If you have no experience in threejs the best resource on the web to get started is the course made by Bruno Simon: ThreejsJourney
If you’re looking for resources on react-three-fiber you can take a look at this repository

Format

There are 2 possibilities to consume this article. You can simply read it to get a global understanding of how the demo works or you can try to reproduce the demo to have a deeper understanding.
If you choose the latter, I created a starter project on codesanbox with all the assets to get started more easily. You can also download it if you prefer to work locally.
Feel free to choose what suits you best.

Starter

Complete demo

flagrede / death-stranding-ui

Death Stranding UI made in React

Death Stranding GameUI demo

Demo link

https://deathstranding.gameuionweb.com/

Article link:

https://dev.to/flagrede/how-to-reproduce-death-stranding-ui-with-react-and-react-three-fiber-cif

Stack

React
react-three-fiber
react-three-a11y
react-spring
twind
drei

Credits

Briefcase model

View on GitHub

The stack

The base project is a classic create-react-app. Here’s the list of the additional libraries used in it:

react-three-fiber (for the 3D scene)
react-spring (for 2D and 3D animations)
valtio (state management)
twind (styling solution based on Tailwind)
drei (react-three-fiber helpers collection)

A note on Twind:
This library is a CSS-in-JS version of TailwindJS. If you're more comfortable with another styling solution don't hesitate to replace it. If you prefer vanilla Tailwind, Twind can be used just like that by using the following shim (already included in the starter).

Interface components

We're going to start building our interface with the 3D part. First, we will create the 3D grid of the private locker. The grid cell delimitations will be done using particles.
Then we will create two smaller grids (for shared locker and sam cargo) without particles. Finally, we need to be able to move the camera between these 3 positions.

3D

Components List

Briefcase

This component will be responsible for loading and displaying the model. We will go through the whole process but some parts are already done in the starter.

download our gltf model from sketchfab (credit goes to luac for the model)
convert it to a react component using gtltfjsx locally or the new online version
convert PNG to JPEG and optimize them
using draco to convert our gltf file to GLB and compress it at the same time.
put the GLB file in our /public folder

At this point, we should be able to see the model. Now we have to position/rotate/scale the model correctly so it fits the original UI.

We will also handle a secondary display for the model. It will be useful later on to separate the selected item from the other. For this secondary display, we will try to display it with a translucent blue color and a wireframe on top of it.

First, we need to duplicate the main material (the first one) of the briefcase into two meshes
For the translucent blue color we can use a simple shader by using component-material on the first material

const SelectedMaterial = ({ blue = 0.2, ...props }) => {
 return (
   <>
     <Material
       {...props}
       uniforms={{
         r: { value: 0.0, type: 'float' },
         g: { value: 0.0, type: 'float' },
         b: { value: blue, type: 'float' },
       }}
       transparent
     >
       <Material.Frag.Body children={`gl_FragColor = vec4(r, g, b, blue);`} />
     </Material>
   </>
 )
}

For the wireframe it’s already built-in threejs, we just have to use the wireframe attribute on the second material

To simulate the selected state you can try to use react-three-a11y. By wrapping our model with the <A11y> component we will have access to hover, focus, and pressed state through useA11y() hook. We can try to display a SelectedMaterial based on the hover state for example.

Since we will have a 2D overlay on top of the 3D scene we won’t need react-three-a11y afterward but it’s good to know that you can bring accessibility to your 3D scene quite easily with it.

Particles grid

This will be the most complex part of the demo.
To recreate this grid we will need 2 components:

A Grid component to display the particles
A GridContainer to compute the positions of the particles and the briefcases

There are 2 different kinds of particles which are called smallCross and bigCross. In the end, we will have to compute these 2 position arrays plus the one for the briefcases.

Grid

First, we will start with the Grid component.

const Grid = ({ texture, positions = [], ...props }) => (
 <points {...props}>
   <pointsMaterial
     size={0.6}
     opacity={0.5}
     color="#316B74"
     alphaMap={texture}
     transparent
     depthWrite={false}
     blending={THREE.AdditiveBlending}
   />
   <bufferGeometry attach="geometry">
     <bufferAttribute attachObject={['attributes', 'position']} count={positions.length / 3} array={positions} itemSize={3} />
   </bufferGeometry>
 </points>
)

Here we’re using an alpha map texture to recreate the “cross” particle effect. We’re also tweaking a few parameters for the colors and the transparency. The particle’s positions and count are given to the bufferAttribute tag. The positions array needs to have the following format [x1, y1, z1, x2, y2, z2, ...].

GridsContainer

Let’s continue with the GridsContainer.
We said that we have 3 position arrays to compute but we can do the 3 of them at the same time.

First question, how many particles do we need for the small cross particles array?

Let’s say we want

20 particles per line
6 lines
2 layers

Also for one particle weed 3 values (x, y, z).
In the end, we will need an array 720 values (20 * 6 * 2 * 3) to display a grid of 20 columns, 6 lines, and 2 layers.

This is only for the small cross particles position array, the big cross array has 2 times less coordinate and the briefcases one 4 times less.

This is because for each cell we want to display:

4 small cross particles
2 big cross particles
1 briefcase

There are probably several ways of doing this. Here’s one method:

loop over the array with 720 placeholder values
for each loop, we need to know if we’re computing an x, y, or z coordinate
for each case, we compute 3 differents coordinates (small cross, big cross, briefcase)
we push these 3 coordinates in their respective arrays

At the end of the loop, we can filter the coordinates we don’t need for the big cross and briefcases arrays (remember that we have 2 times and 4 times fewer coordinates for these too).

Don’t hesitate to put every configuration variable (columns, lines, layers, spacing …) for this grid in a tool like leva to make it look like what you want.

In the actual render, we need to:

map over an arbitrary number (we will change that later)
render our Briefcase components with positionsBriefcases values
render a Grid components with positionsSmallCross values
render a Grid components with positionsBigCross values

External grid

This one is simpler than the grid we just build since it doesn’t use any particles.
Here we just want to display briefcases on the same Z value, 3 columns, and any number of lines. In our new ExternalGrid component we will map just the briefcases list and call a util function to get the position.

Our util function to get the position could look like this:

const X_SPACING = 2
const Y_SPACING = -1

export const getPositionExternalGrid = (index, columnWidth = 3) => {
 const x = (index % columnWidth) * X_SPACING
 const y = Math.floor(index / columnWidth) * Y_SPACING
 return [x, y, 0]
}

Floor and fog

To make the scene look right color-wise on the background we have to add a floor and a fog.

Floor:

   <Plane rotation={[-Math.PI * 0.5, 0, 0]} position={[0, -6, 0]}>
     <planeBufferGeometry attach="geometry" args={[100, 100]} />
     <meshStandardMaterial attach="material" color="#1D2832" />
   </Plane>

Fog:

<fog attach="fog" args={['#2A3C47', 10, 20]} />

Add these 2 elements to the main canvas.

2D

State and data

Before going into building the HTML UI we need to create our state with the data.
For this demo, I wanted to give a try to valtio as the state manager.

We will need to create a state with proxyWithComputed, because we will have to computed values based on the state.

In the actual state we have only two values:

allItems (list of all the briefcases)
selectedItem (index of the selected briefcase inside allItems)

To populate it we need a function to generate data. This function already exists in the starter.

So our state looks like this for now:

proxyWithComputed(
 {
   selectedItem: 0,
   allItems: [...generateItems(9, 'private'), ...generateItems(3, 'share'), ...generateItems(3, 'sam')],
 },

The second parameter takes an object and is used to define the computed values.
Here’s the list of computed values we will need:

isPrivateLocker (based on the selectedItem)
isShareLocker (based on the selectedItem)
isSamCargo (based on the selectedItem)
itemsPrivateLocker (filter allItems)
itemsShareLocker (filter allItems)
itemsSam (filter allItems)
allItemsSorted (use filter computed values to sort the array)
selectedId (ID of the selected item)
selectedCategory (category of the selected item)
totalWeight (sum of briefcase weight inside Sam cargo)

Components List

Inventory

This is the component that will display our list of briefcases. As we saw on the schema it uses the following child components:

MenuTab (pure UI component)
MenuItems (display a portion of the list, ie: briefcases in PrivateLocker)
ActionModal (will be discussed just after)

The component should also handle the following events:

keyboard navigation
mouse events
update the selected briefcase in the store
open ActionModal

Action modal

In this modal, we add actions to move the selected briefcase from one category to another.
To do that we just need to update the category of the selected item in the store. Since we’re using computed values to display the lists, everything should update automatically.

We will also need to handle keyboard navigation in this modal.

Item description

This is the right side part of the UI. We just need to display all the data of the selected item here.

The only interaction is about the like button. Whenever the user clicks on it we should update the likes count of the selected briefcase. This is simple to do thanks to Valtio, we just update allItems[selectedItem].likes in the state directly and the likes counts should update in the Inventory.

Combining 2D and 3D

We now have a 2D UI and a 3D scene, it would be nice to make them interact with each other.

Selected briefcase

Currently, we just highlight the selected item in the UI part. We need to reflect this to the 3D briefcase as well. We already made the selected material, we just need to use it inside the Briefcase component.

Scene transition

From now on, our camera was only looking at the main grid, the private locker. We will create 3 components to move the camera and display them based on the properties isPrivateLocker, isShareLocker, and isSamCargo that we created earlier in the state.

Here for example the code that look at the main grid:

function ZoomPrivateLocker() {
 const vec = new THREE.Vector3(0, 1.5, 4)
 return useFrame((state) => {
   state.camera.position.lerp(vec, 0.075)
   state.camera.lookAt(0, 0, 0)
   state.camera.updateProjectionMatrix()
 })
}

Adding perspective

To give our UI a more realistic look we must make it look like it is slightly rotated from the camera. We can do that with the following CSS:

body{
  perspective 800px;
}

.htmlOverlay {
  transform: rotate3d(0, 1, 0, 357deg);
}

Animations

We’re now going to add some animations to both the UI and the 3D scene.
All animations has been done using react-spring.

2D

MenuEffect

This is the animation that happens inside Inventory whenever the selected item changes.

There are actually 3 parts to this animation:

a sliding background going from left to right
the item background going from 0 to 100% height
a slight blinking loop for the background-color

We will go through each of them and combine them together with the useChain hook.

Sliding animation

To reproduce this animation we will need custom SVGs (they are already available in the starter). I used the tool https://yqnn.github.io/svg-path-editor/ to make 3 SVGs.

I think we could have an even better effect with more SVGs, feel free to try adding more frames to animation.
To animate these 3 SVGs, we will declare a x property inside a useSpring going from 0 to to 2 and in the render we will have this:

         <a.path
           d={
             x &&
             x.to({
               range: [0, 1, 2],
               output: [
                 'M 0 0 l 16 0 l 0 3 l -16 0 l 0 -3',
                 'M 0 0 l 25 0 l -10 3 l -15 0 l 0 -3',
                 'M 0 0 l 16 0 L 16 3 l -5 0 l -11 -3 m 11 3',
               ],
             })
           }
         />
       </a.svg>

Now we just need to animate the opacity and the width and we should have a good sliding animation effect.

background height

Here we’re just expending the item’s background with a default spring:

const [{ height }] = useSpring(() => ({
   from: { height: 0 },
   to: { height: 24 },
   ref: heightRef,
 }))

glowing color animation
To reproduce this part we will make a spring between 2 colors and play with the opacity at the same time:

 const [{ bgOpacity, color }] = useSpring(() => ({
   from: { bgOpacity: 1, color: '#456798' },
   to: { bgOpacity: 0.5, color: '#3E5E8D' },
   ref: bgOpacityRef,
   loop: true,
   easing: (t) => t * t,
   config: config.slow,
 }))

All together
Finally, we just have to use these 3 animations with the useChain hook

 useChain([opacityRef, heightRef, bgOpacityRef], [0, 0.2, 0])

SideMenuEffect

The SideMenu animation will use the same technique we just saw. It will be a spring that goes through 3 SVGs. Again I was a bit lazy on the number of SVG frames, feel free to try with more.
Here are the 3 SVGs I used for the demo:

             output: [
               'M 0 0 l 16 0 l 0 3 l -16 0 l 0 -3',
               'M 0 0 l 25 0 l -10 3 l -15 0 l 0 -3',
               'M 0 0 l 16 0 L 16 3 l -5 0 l -11 -3 m 11 3',
             ],

AnimatedOuterBox

Here our OuterBox component:

const OuterBox = () => (
  <div>
    <div className="h-1 w-2 bg-gray-200 absolute top-0 left-0" />
    <div className="h-1 w-2 bg-gray-200 absolute top-0 right-0" />
    <div className="h-1 w-2 bg-gray-200 absolute bottom-0 left-0" />
    <div className="h-1 w-2 bg-gray-200 absolute bottom-0 right-0" />
  </div>
)

This component is displayed inside ItemDescription one. It shows four little white stripes at the edges of ItemDescription.

On the animation side, we will have to animate the height property of the component from 0 to 100%.

AnimatedBar

For the bar that shows an item's durability, we will make an animated bar (like a loader).
We need to animate the width property based on the damage attribute of the item.

3D

For the 3D scene, we will add just one animation that will be triggered whenever a briefcase is changed from one category to another. We will make it seem like the briefcases, those that have changed, are falling from above.

We can handle this animation in the Briefcase component. Whenever the position of a briefcase will change, we will animate the new value on the Y-axis from the new value plus a delta to the new value.

Until now the spring animations were triggered whenever a component was mounted. Here we need to animate briefcases that are already mounted.
To trigger a spring that has already been played once we need the second parameter received from the useSpring hook.

  const [{ position: animatedPosition }, set] = useSpring(() => ({
    from: { position: [position[0], position[1] + 5, position[2]] },
    to: { position },
  }))

Be careful to use @react-spring/three instead of @react-spring/web here.

Sounds

For the sounds part we’re going to create a sound manager component using useSound hook from Joshua Comeau. After that, we’re going to put our sound functions newly-created into our state so that we can everywhere in the app.

Here’s the list of sounds we need to handle:

like button
menu change (played whenever the item selected change)
menu action (played whenever the action modal is opened)
menu validate (played whenever the action modal is closed)

Conclusion

We’re done with the tutorial, I hope you liked it. If you’re trying to make your own version of the Death Stranding UI, don’t hesitate to share it with me on twitter. If you're interested in more GameUI on web demos I share updates on the upcoming demos on this newsletter.

Making a 2D RPG game with react-three-fiber

Florent Lagrede — Thu, 22 Oct 2020 06:52:28 +0000

In this article we're going to take a closer look at an open source demo published by @coldi. Coldi made a game, called Colmen's Quest (that you should definitely check out), using react and react-three-fiber. He was kind enough to share the core engine that he made for his game to the community.

It might sound odd to use a 3D library like ThreeJS to make a 2D game but it is actually not that uncommon at all. For example Unity, the popular 3D game engine, is also used a lot for 2D games like Hollow Knight.

Speaking about Unity, the game architecture that Coldi used is also inspired by Unity and resolve around the concept of GameObject components that we will talk about just after.
Adding react-three-fiber to the stack provides a terrific dev experience to make a webgl game with React.

This project is a really valuable learning material. By exploring it in this article we will learn a lot about game dev techniques, react-three-fiber and also React knowledge in general. We will also try to apply our newly acquired knowledge by tweaking the demo a bit. Let's dive in !

The game demo

Demo link

Let's start by analyzing the elements and features that we have in this demo.
We have:

🗺 A map
- defined with tilesets
🚶‍♂️ A character that can be moved with either a mouse or a keyboard
- the mouse movement is trickier as it needs to compute the path ahead
🧱 A collision system
- which prevents to walk into walls or objects
👉 An interaction system
- pizza can be picked up and it is possible to interact with computers and coffee machines
📽 A scene system
- to move from one room to another

We can start by cloning the demo here:

coldi / r3f-game-demo

A demo on how to do a simple tile-based game with React and react-three-fiber

react-three-fiber Game Demo

This repo shows an example implementation of a top-down 2d game made with React and react-three-fiber.

I used the core functionality to create Colmen's Quest and wanted to give you an idea of how a game can be done with React.

This is by no means the best way to build a game, it's just my way. 😊

I suggest you use this code as an inspiration and not as a starting point to build your game on top of it. I also do not intend to maintain this code base in any way.

Get started

You can start the game by yarn && yarn start, then open your Browser.

To get a better understanding of the architecture I used, you may want to read this thread on Twitter.

👉 Also Florent Lagrede (@flagrede) did an amazing job in writing an…

View on GitHub

Folders architecture

@core: everything that is reusable and not specific to the current demo
components: components that hold logics more specific to the current demo.
entities: describe elements in the game world (Pizza, Plant, Player...). All these elements are GameObject. We're going to explain more of this concept just below.
scenes: represents the different rooms in the game. Scenes are an aggregation of GameObject. In the demo there are two scenes (Office and Other).

Game architecture

The component architecture looks like this:

    <Game>
        <AssetLoader urls={urls} placeholder="Loading assets ...">
            <SceneManager defaultScene="office">
                <Scene id="office">
                    <OfficeScene />
                </Scene>
                <Scene id="other">
                    <OtherScene />
                </Scene>
            </SceneManager>
        </AssetLoader>
    </Game>

We're going to explain each of them.

Architecture - top part

Game

This component has 4 main features:

register all GameObject inside the game
a global state
render the Canvas component from react-three-fiber
pass a context to all its children with the global state and methods to find/register GameObject

AssetLoader

This component will load all image and audio assets of the game with the Image and Audio web object. It also displays an html overlay on top of the canvas while the assets are loading.

SceneManager

This component holds the state regarding the Scene currently being displayed. It also exposes a method setScene through a Context in order to update the current scene.

Scene

This component, besides displaying its children GameObject, will dispatch the events scene-init and scene-ready whenever the current scene changes.

There is also a level system present in the file that is not being used by the demo.

Architecture - Bottom part

Now we are going to look a little deeper, inside the code of the OfficeScene.

    <>
        <GameObject name="map">
            <ambientLight />
            <TileMap data={mapData} resolver={resolveMapTile} definesMapSize />
        </GameObject>
        <GameObject x={16} y={5}>
            <Collider />
            <Interactable />
            <ScenePortal name="exit" enterDirection={[-1, 0]} target="other/start" />
        </GameObject>
        <Player x={6} y={3} />
    </>

The GameObject component we saw earlier is the most important piece of the architecture. It represents almost every element in the game world. For example for the OfficeScene just above we have 3 GameObject:

A Map
A Scene changer
The Player

GameObject holds state information like position, enabled/disabled or its layer in the game (ie: ground, obstacle, item, character ...). They can contain other GameObject as well.
GameObject can also contain other components that Coldi called Scripts. These scripts can hold the logic for interaction, collision or movement for example. Basically game objects are a composition of these reusable Scripts and other GameObject. This is a really powerful API because you can describe a game object behaviour component by just dropping components in it.

Game Objects

We're going to explore more the 3 GameObject we saw earlier:

The map

This component will create the map of the Scene based on an entities mapping string. For example the Office mapping string looks like this:

# # # # # # # # # # # # # # # # #
# · W T # T · · W T · W · · · T #
# · · · · · · · · · · · · · · o ·
# o · · # · · · # # # # · · # # #
# # # # # · · · # W o W · · T W #
# C C C # · · · T · · · · · · · #
# o · · · · · · · · · · · · · o #
# # # # # # # # # # # # # # # # #

Inside the OfficeScene there is a function called resolveMapTile which will map each character to a game entity. Entities are GameObject that match a real element in the game world.
In this case we have the following entities mapping:

# : wall
. : floor
W : workstation
C : coffee machine
T : plant

The child component TileMap will then be responsible to return the map base on the entities mapping string and the resolveMapTile functions.

The final map is a 2D grid, with each cell holding one or several GameObject components.

Entities - workstation

Let's take a closer look at what an entity looks like. We're going to look at the Workstation one.

export default function Workstation(props: GameObjectProps) {
    return (
        <GameObject {...props}>
            <Sprite {...spriteData.objects} state="workstation-1" />
            <Collider />
            <Interactable />
            <WorkstationScript />
        </GameObject>
    );
}

We can see the GameObject component we were talking about and some child components(Sprite, Collider, Interactable and WorkstationScript) that define its behaviour.

Sprite

The Sprite component is responsible for displaying all graphical elements in the game.
We didn't talk much about react-three-fiber until now, but it is in this component that most of visual rendering happens.

In ThreeJS elements are rendered through mesh objects. A mesh is a combination of a geometry and material.

In our case for the geometry we're using a simple Plane of 1x1 dimension:

THREE.PlaneBufferGeometry(1, 1)

And for the material we're just applying the Threejs basic material:

<meshBasicMaterial attach="material" {...materialProps}>
    <texture ref={textureRef} attach="map" {...textureProps} />
</meshBasicMaterial>

With a plain basic material however we would be just seeing a simple square. Our sprites are actually displayed by giving the <texture> object, which will apply sprites to the <meshBasicMaterial>.

To sum up, the visual render of this demo is mostly 2D plane with texture applied to them and a camera looking at all of them from the top.

The collider

This component is responsible for handling collisions. It has two jobs:

store the walkable state (if it is possible to step on it or not) of the GameObject using it. By default the Collider is initialized as non walkable.
listen and trigger events to do some logic whenever there is a collision.

The component also uses the hook useComponentRegistry to register itself to its GameObject. This allows other elements in the game (like the player) to know that this game object is an obstacle.

For now we just have added an obstacle on the map, let's continue with the next component.

Interactable

This component is responsible for handling logic when the player interacts with other elements in the game. An interaction occurs when the player has a collision with another GameObject (this is why the Collider from earlier was needed).

Interactable has severals methods:

interact: executed by the GameObject that initiates an interaction
onInteract: executed by the GameObject that receives an interaction
canInteract: is it possible to interact with it

The Interactable component, as the Collider, registers itself to its GameObject.

The WorkstationScript

function WorkstationScript() {
    const { getComponent } = useGameObject();
    const workState = useRef(false);

    useGameObjectEvent<InteractionEvent>('interaction', () => {
        workState.current = !workState.current;

        if (workState.current) {
            getComponent<SpriteRef>('Sprite').setState('workstation-2');
        } else {
            getComponent<SpriteRef>('Sprite').setState('workstation-1');
        }

        return waitForMs(400);
    });

    return null;
}

At last we have a script, specific to this entity, to handle some logic.
We can see here that this script is listening to the interaction event from earlier. Whenever this happens it just swaps the sprite of the computer.

Exercice

We're going to add a monster entity, disguised as a plant. Inside the object sprite sheet asset, we can see that there are two plants that are not used in the demo.
The goal will be to use them to create a new entity called ZombiePlant and place it inside the other Scene.

When interacting with the entity, the plant should swap from one sprite to the other one.

We will also have to change both the entities mapping string and the resolveMapTile function inside the OtherScene.

Solution

The scene changer

        <GameObject x={16} y={5}>
            <Collider />
            <Interactable />
            <ScenePortal name="exit" enterDirection={[-1, 0]} target="other/start" />
        </GameObject>

Now let's look at the components that handle the scene change.
This component will be triggered when the player steps on it.

To create this effect, the scene changer has 3 child components:

Collider
Interactable
ScenePortal

We are already familiar with some elements like Interactable and Collider. This shows us how reusable GameObject can be with this architecture. Let's look at the ScenePortal.

Scene Portal

This component is responsible for doing the scene change when the player interacts with it.
It has the following props:

name: name of the portal
target: destination where the player should be teleported (scene + portal). This parameter is a string with the following template: sceneName/portalName
enterDirection: direction that the player should face when entering the new scene;

The component listens to the interaction event through the hook useInteraction. When he receives an interaction, it will check if it comes from the player. In that case the port function is called. It will change the current scene in the global game state. After that the component will wait for the SceneInitEvent and SceneReadyEvent to move the player in the right position and direction.

Workflow example

Let's try to visualize the whole workflow of the ScenePortal:

The Player

We're now going to explore the biggest GameObject of the game, the Player one.
The player GameObject looks like this:

    <GameObject name="player" displayName="Player" layer="character" {...props}>
        <Moveable />
        <Interactable />
        <Collider />
        <CharacterScript>
            <Sprite {...spriteData.player} />
        </CharacterScript>
        <CameraFollowScript />
        <PlayerScript />
    </GameObject>

We are still familiar with Interactable and Collider.
Let's see what the new components are doing.

Moveable

This component just exposes an API, it does not listen to any events. It means that there will be another GameObject that will call the Movable's API to move the GameObject using it (in our case the Player).

The most important method is the move one. It takes a targetPosition as parameter, checks if this position is a collision and if not move the GameObject to it.

It also triggers a lot of events that can be used elsewhere. The events sequence look like that:

Also the method move uses the animejs library to animate the player sprite from one position to another.

CharacterScript

    useGameLoop(time => {
        // apply wobbling animation
        wobble();

        // apply breathe animation
        if (!movementActive.current) {
            // breathe animation while standing still
            const breathIntensity = 20;
            scaleRef.current.scale.setY(1 + Math.sin(time / 240) / breathIntensity);
        } else {
            // no breathe animation while moving
            scaleRef.current.scale.setY(1);
        }
    });

This component is responsible for doing some animation to the Player Sprite. The script handle:

flipping the sprite in the current moving direction (use the attempt-move event we saw earlier)
apply a wobble effect while moving
- this effect is applied inside the useGameLoop hook. Under the hood this hook uses the useFrame hook from react-three-fiber. This hook is really useful as it allows us to perform update on each frame
add a footstep sprite and sound while moving
make the animation bounce while moving (use the moving event we saw earlier)

To sum up this component perform sprite animation by listening to movement events from the Moveable component.

PlayerScript

Final piece of the Player entity, the PlayerScript.
This component handles the logic that the player can do. It will deal with both cursor and keyboard inputs.

Keyboard controls

There are 4 hooks useKeyPress that add the listener to the key given in parameter. These hooks return a boolean whenever the listed keys are pressed. These booleans are then checked inside a useGameLoop, that we saw previously, and compute the next position consequently. The new position is set in the local state of PlayerScript.

Cursor controls

This part is a bit more tricky. While the keyboard controls could move the player one tile by one tile, the cursor can move it to several tiles. It means that the whole path to the selected position must be computed before moving.

In order to do that the method use a popular path finding algorithm named A star (or A*). This algorithm computes the shortest path between two points in a grid by taking collision into consideration.

As for the keyboard events the new position is updated into the local PlayerScript state. In addition the path is also displayed visually in this case. In the render method there is PlayerPathOverlay component which is responsible for doing just that.

Moving to the new position

In both cases we saw that the new position is updated in the local state of the component.
There is a useEffect that listens to that change and that will try to move the GameObject. Remember the Moveable component from before ? Here we get it and call its move method on him. If the move is not possible, the method returns false. In that case we will try to interact with the GameObject that is in the position that the player couldn't go to.

Exercice

This was a big piece but now we should understand how game objects work together, let's try to make a new thing now.

Remember our ZombiePlant entity? We're going to add some new features to it:

When the player interacts with it: should bounce back from the player (like if the player was attacking it)
Whenever the interaction occurs: should play a sound effect (we can reuse the eating for example)
On the third interaction the zombie plant should disappear

Solution

Conclusion

This is it, we've gone through most of the demo !
I hope you learned a lot of things in this demo walkthrough (I did). Thanks again to @coldi for sharing this demo with the community.
Also as he said a lot of things could have been implemented differently. For example the collision system could have been done with a physic engine like react-use-cannon.
This is still a terrific example of how to make games with react-three-fiber.

Hopefully this gives you some ideas to make a game of your own !

If you're interested in front-end, react-three-fiber or gamedev, I will publish more content about these topics here.

Thanks for reading, happy coding.

How to replicate the Zelda BOTW interface with React, Tailwind and Framer-motion: Part 3

Florent Lagrede — Sun, 19 Jul 2020 09:25:52 +0000

For the final part of the Zelda series we're going to add a lot of little details, more animations and sounds !

Checkpoint

After the second part our interface should look like this.

Items actions

Creating the modal

We are going to start by creating a Modal component in order to interact with the item selected. The modal will provide the equip and drop actions. For this step, we will display the modal when either a click or the Enter key is pressed on a selected item. The actions will be handled later on, we can close the modal instead when they are selected. We will also handle keyboard navigation inside the modal actions and create a useClickOutside hook to close the modal when we click elsewhere.

Notes

We also must not forget to reset the focus when the modal is closed to keep the interface navigable with the keyboard.

Solutions

In the first commit below, only the keyboard is handled to open the modal. The onClick handle is done in the second commit.

Handling modal actions

We are now going to handle the equip and drop actions inside the modal.
For the equip feature we have some requirements. Link can have equip at the same time the following items:

1 Weapon
1 Shield
3 Armors:
- 1 Helm
- 1 Armor
- 1 Greaves

We can also add some visual indicators to show that an item is equipped like a radial background in the item cell and a blue bar inside the ItemInformation component. The blue color should be defined in our Tailwind config file.

For the data structure of our new state we can use for instance an object with the item.category string as key and the item data as value.

{
  [key: string]: ItemType
}

The drop feature should be simpler, we just want to remove the item selected and add an empty item at the end instead.

Solutions

Checkpoint commit

Handling items with special bonus

We are now going to handle the bonus for armor's items. To do so we have to:
1) Display the bonus icon of an item inside its cell
2) Compute all the active bonus based on what items are equipped
3) Display the active bonus, with a new component, inside the right column

For the first step we need to create a BonusIcon component which takes for example a bonusType as props and returns the appropriate bonus icon (fire, swimming or climbing). Then we can use this component inside the Item component.

For the second step we need to create a state to store the active bonus. These bonuses will be updated whenever the items equipped change. We can use a useEffect hook for that. For the data structure of our new bonus state it could look like that (if you are using TS, otherwise define key as usual):

{
   [ItemsBonusType.FIRE]: 0,
   [ItemsBonusType.SWIMMING]: 0,
   [ItemsBonusType.CLIMBING]: 0,
}

For the final step we are going to create two new components BonusBox and BonusList.

BonusBox will display a BonusIcon and little bars, up to a max of 3, to show how many items which have this bonus type are equipped.
BonusList will receive the data we've created on step 2 and display a bonus for each of the bonus types.

Solutions

Checkpoint commit

Adding little details

Sounds

To add sounds to our interface, we are going to use the handy use-sound hook from Joshua Comeau.
The sounds has been added in the repository. There is one for selection (arrow keys, click) and another one for actions (equip, drop). The sounds should be playable from anywhere inside the app so we should create a new context for that. We might need to create a wrapper for our main App component for the new sound context.
Then we should use the sounds wherever it feels necessary.

Solutions

Checkpoint commit

Endurance gauge

This is going to be a small step but details matter. We are going to display an endurance gauge with css only.
We are going to need two divs, one for the outer circle and one for the inner circle. To create the circle effect we just just have to apply a border-radius: 50% to them.

Notes

It might be easier to create a custom css file for this rather than add all missing values to Tailwind, since we're not going to need them elsewhere.

Solutions

Checkpoint commit

Full durability star

For the final touch, we are going to add a sparkle animation for items with full durability (items with the isNew property in the dataset).
Like we've done before, we need to convert the sparkle SVG from the asset folder with SVGR.

After that, we need to create an animation, still with framer-motion, to rotate and scale the sparkle at the same time and infinitely.

Ressources

Joshua Comeau has written a great article on the subject.

Solutions

Checkpoint commit

Wrapping up

Wow, we're finally done with the ZeldaUI, congrats on completing it!
I hope that you learned new things and that you had fun at the same time.
If you have any ideas or feedback, let me know in the comments. I will gladly try to address them.

See you then!

How to replicate the Zelda BOTW interface with React, Tailwind and Framer-motion: Part 2

Florent Lagrede — Sun, 05 Jul 2020 07:54:20 +0000

Introduction and notes on first part

Welcome back for the second part of the Zelda series.
I hope that you manage to complete the first part without much trouble. If not or if you've missed the first part I created a checkpoint branch with all the steps completed.
Also there were some errors in the solutions provided in part 1 that should all be fixed now.

After the first part, our interface should looks like this:

In this second part we'll focus first on the right column of our layout and then we're going to handle items pagination and ways to navigate between them.

For providing solutions I decided to stop using gist files, they were hard to maintain and even for reading it was not convenient either to jump between files for the same solution.
Instead I created a checkpoint commit at each step of the second part, it should make reading easier hopefully.

Displaying additional items information

Right column layout

We're going to start by creating the base blocks of the right column.

We need an ItemInformation component that should be displayed at the bottom of the column. For now the component should be an empty box, but its width should match the ItemGrid width when we are in column display.

Once the ItemInformation component is set, we'll need to position the Link image below and hide it for breakpoints below XL. We can use an absolute here for the positioning.

Notes

The color for the ItemInformation background is missing in the starter tailwind configuration file. We can add it now:

bgBlackTransparent: "rgba(0, 0, 0, 0.5)",

Solution

Checkpoint commit

Items information

Now we're going to display more Item information. This step is mostly passing down props and implementing the design of the ItemInformation component.

We want to display the name, value, category and description of the item. The category will be displayed as an Icon. In the first part we've displayed only one category among the 3 available in the dataset but we'll handle all 3 cases here. We need to create a CategoryIcon component which displays the appropriate icon based on a type props for example.

But first we have to convert our SVGs (Armor, Shield, Sword) inside src/assets/items into React components. We will use SVGR for that.

On this step we will also add the calamity font since this is the first time we will display text. It is available in the font folder under assets. We have to move it to create-react-app public folder and add font-face declarations in a separate css file.

Notes

We can reuse the Armor icon for helm and greave category.

Ressources

https://developer.mozilla.org/fr/docs/Web/CSS/@font-face

Solution

Checkpoint commit

Adding a component to animate text

Now we are going to animate the item description by adding a typewriter effect animation.

To achieve this effect we are going to use the windups library from @sgwil. The library is really well written, has a nice API and great documentation. We can even trigger effects, like sounds, during the animation. For this demo we are just going to use the base functionality of windups with the useWindupString hook provided for React.
This hook renders a lot of time so we should avoid using it at a high level in our app structure otherwise it would trigger a lot of unneeded render on children components and seriously lower the performance of our interface.
Instead we will create a TypeWriter component that we will use inside ItemInformation.

Notes

Have a look at the interactive guides to get a feeling of what you can do with windups or even the sources if you have some time they are really interesting.

Ressources

useWindupString doc

Solution

Checkpoint commit

Adding pagination

Handling pagination logic

In the first part we create a getItems function to retrieve one category of items and fill the rest of the grid cells with an empty item object.

Now we have to tweak this function a bit to return all the categories of items by page.

The function should have the following ItemsPage return type


export enum ItemsMainCategoriesType {
  WEAPONS = "weapons",
  SHIELDS = "shields",
  ARMORS = "armors",
}

export type ItemsPage = {
  items: ItemType[];
  mainCategory: ItemsMainCategoriesType;
  page: number;
};

Once that is done we need to create two states using useState react hook inside our App component:

One that should be initialized with the result of our getItems function
Another one to store the current page, that we initialize to 0 for now
Then we need to create our new items variable based on the active page

At the end of this step we won't be able to navigate between pages yet but we'll have the base logic ready.

Solution

Checkpoint commit

Creating a component for page navigation

For the next part we're going to create the component to move between the groups of items, we can call it CategoriesMenu. We'll also need a component for each item inside that menu, we can call it CategoriesMenuItem.

The CategoriesMenuItem will have to display one of the 3 SVG react components (Weapon, Shield, Armor) and call the setPage from earlier on click. In the image, the first icon is selected and has different style applied.

Solution

Checkpoint commit

Adding animated arrow to navigate

We are now going to provide a second way to navigate between the pages with arrows at the edge of the items grid component.

This time instead of calling setPage directly, we'll create a navigateToDirection function which should either take 1 or -1 to represent the direction and then call setPage with the appropriate value.

These arrows should:

call navigateToDirection
be animated on the scale property to create a zoom in/out effect with framer motion
be disabled if reaching either max/min page.

We'll also have to update our useState inside App to store both the page and the direction.

Notes

The arrow image is available under the asset folder under the name arrow-no-curve.png. We may have to move it to the public folder to use a static url directly.

We only have one arrow as an image, so we need to rotate the image to create the left arrow.

Solution

Checkpoint commit

Handling keyboard to navigate between page

We are now going to provide a third way to navigate between the pages ! Triforce requirement, we have no choice.

For this step we have to change the handleKeyPressed and goLeft / goRight functions we wrote in part 1.

handleKeyPressed: we need to call navigateToDirection inside ArrowLeft and ArrowRight event when we are at the edges of the ItemsGrid
goLeft / goRight: we loop over horizontal cells when we reach the edges of the ItemGrid.

Notes

We can optionally refactor the if/else inside the handleKeyPressed function by a switch block.

Solution

Checkpoint commit

Adding animation on page change

For the last part we are going to add an animation to the page transition.

The animation will be added to the ItemsGrid component. We want to slide the ItemsGrid from the right or from the left depending on the direction that we stored in our useState.
To create this animation we are going to use the variant api from framer-motion, it allows us to define the different states of our animation in an object. In this object we will have two states:

enter: this is the starting position, here we will slide the ItemsGrid to either -100 or 100 on the x axis and apply an opacity to 0.
center: here we will move the ItemsGrid back to the center and reveal the ItemsGrid at the same time with opacity to 1.

To sum up we are going to give our motion.div a variant object with two entries enter and center. Each entry will define values for x and opacity properties. It should look like this:

const variants = {
  enter: (direction: number) => {
    return {
      x: ??,
      opacity: ??,
    };
  },
  center: {
    x: ??,
    opacity: ??,
  },
};

We'll also need to define the transition we want for these properties, we can use the values below:

      transition={{
        x: { type: "tween" },
        opacity: { duration: 0.2 },
      }}

In the end our motion.div component inside ItemsGrid should have the following props:

<motion.div
    custom={direction}
    variants={variants}
    initial="enter"
    animate="center"
    transition={{...}}
/>

Since there is no direction involved when we change pages using the CategoriesMenu we should probably just animate the opacity in this case.

Ressources

Variant API

Solution

Checkpoint commit

What's next

Congratulations for completing part 2 we've made a lot of progress on completing our demo.
In part 3 we will add the final touch to bring this interface to life !
Again feel free to share any feedback and I hope you had fun.

How to replicate the Zelda BOTW interface with React, Tailwind and Framer-motion: Part 1

Florent Lagrede — Sat, 20 Jun 2020 14:22:40 +0000

In this article series we'll learn how to replicate the menu inventory interface of Zelda: Breath Of The Wild on the web !

Click on the image below to check the final demo:

Articles structure

This will be a series of 3 articles:

PART1: Setup the Item Grid + Adding navigation + Selection Animation
PART2: Adding pagination + page transition animation + display additional items data
PART3: Handling actions with a modal + Handling items with bonus + new item animation + add sound

Target audience:

Intermediate Front-end developer.
If you are more experienced you might not learn much (at least for part 1) but it might still be an interesting exercise to do.

Why doing that ?

I find GameUI very inspiring, they share different problems than web interfaces and also similar ones. They are usually designed to not get in the way of a player's experience of a game (Zelda BOTW is a perfect example of that). Web is not much different in that regard. When we design a website we're actually building an experience, the interface is just the way to interact with the experience.
Also GameUI are often filled with a lot of little details, these details are not always meant to be noticed at first glance yet they amount for a significant part of an enjoyable interface.

About this interface

screenshot from interfaceingame.com

The interface is basically a two layout column.

The first column contains a grid of items grouped by categories.
The second column is used to display information about the item selected.

There are a lot of animations on the interface that all serve a purpose:

arrows moving in and out the edge of the selected items
arrows at the horizontal edge of the grid to indicate that you can navigate between categories
when you change the selected item, the item description is typed progressively to catch your eye on the fact that these data have changed
items that are new (max durability) have a little shiny star. This animation is really subtle compare to the other, because it concerns a detail about the item

The first 3 animations, which are the most visible, act as signifiers (This a UX term that comes from Donald Norman, the author of The design of everyday thing), and helps the player understand the actions he can do on the interface.

About the format

At first I thought of just providing some code samples with explanations but I don't think it would have provided much value.
You might have come to realize that we, as developper, learn best by actively doing something. That's why this series will actively encourage you to code your own version of the Zelda BOTW interface.
I'll give you a goal to reach at each step and provide leads for realizing it.
If you're stuck or if you just want to keep reading, the solutions will be provided at each step in separate gists.
I have setup a based repository with all the ressources to get you started immediately:

flagrede / zelda-botw-starter

Template for reproducing the Zelda BOTW inventory interface

Zelda BOTW starter

Complete demo

https://www.gameuionweb.com/zelda-botw

Articles links:

Stack

React
Tailwind
Framer-motion (beta)

Useful links for Tailwind:

Credits

Items Data: Zelda Gamepedia
Font: Reddit user /75thTrombone

View on GitHub

The repository is set up with Typescript but no obligation here, you can use either ts or js as you prefer.

Prior requirements and notes about technologies used

This articles series is based on the following stack:

ReactJS and leverage some libraries of its ecosystem
- Some prior knowledge is expected but nothing advanced
TailwindCSS
- No prior knowledge expected, we'll introduce it briefly if you are not familiar with it.
Framer-motion:
- This is the main library used for animation, no prior knowledge expected

Tailwind quick introduction

You can skip this part if you're already familiar with Tailwind

Tailwind is a CSS utility framework, it provides a set of css classes with low responsability like w-full for width: 100% or flex for display: flex.

These classes are generated based on a JS configuration file. In this file you define design tokens like color, spacing, ... and then tailwind will generate all the css classes based on the design constraints you defined.

You can find the structure of this file here: Default configuration

I choose Tailwind for the article series because it allows me to prepare the design configuration upfront so you can focus on the most interesting part.
Also I will be able to put tailwind classes needed directly on schemas provided in the article to make implementation even easier.

If this is your first time with tailwind you could use this cheat sheet to help you find class at the beginning: https://nerdcave.com/tailwind-cheat-sheet
I also strongly recommend installing a plugin to get autocompletion on tailwind classes (see starter readme for the links).

Starting with the layout

We're going to start by creating the base layout of our interface.
What we want here is a responsive layout with the following constraints:

Extra large screen (XL): a two columns layout column, with each column taking 1/2 of the screen.
below extra large screen (<XL): there should only be one column taking full width.
The layout should be contained inside a centered responsive container.

Once your layout is done you can add the bg-zelda-darkGreen class to your main div and we're good for that part.

Ressources:

Solution:

https://gist.github.com/flagrede/e440ac5c56f21ef9b9db37d1c780ad20

Adding the items grid

We'll now create our ItemsGrid and Item components.

We want to create a 5 columns Grid, with css-grid, which reduces to 3 columns on mobile, that displays an Item component in each cell.
To create the Item component we will need to take a look at the data available in src/data/items.js. An item data looks like this:

    {
      name: "Tree Branch",
      category: "weapon",
      icon: "/zelda-botw/items/weapons/BotW_Tree_Branch_Icon.png",
      value: "2",
      isNew: true,
      description:
        "Wooden branches such as this are pretty common, but it's surprisingly well-balanced. It doesn't do much damage but can serve as a weapon in a pinch.",
    }

For now our component will just have to display its icon and value and also his name for the image title.
Our item should look like this:

There are 3 main categories of items in the dataset weapons, shields and armors. We need to display only one category for now so we can start by taking the weapons items for example. We can create a getItems function to retrieve that group and fill the rest of the array, with an empty item object, to reach the number of cells wanted.

Ressources:

https://tailwindcss.com/docs/grid-template-columns/

Solution:

Items selections

Now we want to be able to select an item by clicking on it by applying the following classes.

Our app structure should look like this at this point:

<App>
  <ItemsGrid>
    <Item>

We will need to:
- store the index position of the selected item using React.useSate in App component
- create a context with React.createContext to store the selectedItem position and the setSelectedItem from the previous useState
- Retrieve the context value inside the Item component using useContext hook.
- Activate the selected classes from above using classnames utils and the value from the context.

Ressources:

Solution:

https://gist.github.com/flagrede/d0627032cb459166dd454110337a1ab1

Keyboard navigation

For this part we will have to convert the index position of an item to a 5x4 matrix position and another one to revert it.
The grid position should be like this:

So we will need a method that convert for instance 6 to {x:1 ,y:1}
And another method that convert {x:1 ,y:1} to 6
The first method signature should be:

(index: number) => {x:number, y:number}

The second method signature should be:

({x:number, y:number}) => number

We will then need four methods to compute the new position base on the direction goUp, goRight, goLeft, goBottom
I strongly recommend doing tests for these methods, but no obligation here.

Test example:

it("should convert 6 to x:1, y:1 position", () => {
  // Given
  const index = 6;

  // When
  const result = convertIndexToPosition(index);

  // Then
  expect(result).toEqual({ x: 1, y: 1 });
});

Finally a handleKeyPressed function inside the App component to handle the behaviour to execute based on the key pressed.

Note: the keyboard navigation is expected to work only for the 5 columns grid (since we have 3 columns for the mobile layout).

Ressources:

https://reactjs.org/docs/events.html#keyboard-events

Solution:

keyboardNavigation: https://gist.github.com/flagrede/dbce8e6be44e39f37a9f8e459a39f28c
App: https://gist.github.com/flagrede/58c2bd6dfceda4861be0da90cf099c50
Tests: https://gist.github.com/flagrede/72d64e3db9634d5b4cc10f37a9f220ee

Animation

To close this first part we'll add some animation with framer-motion.
We will now add triangles at the edge of the selected item and animate them.
The triangles will be done using css. The css classes to create the triangles are already available, you have zelda-botw-triangle-up and zelda-botw-triangle-bottom.

We will need to create two components:

A Triangle component with the following props:

type Props {
  animateParams: {
    rotate: string, 
    x: [number, number, number],
    y: [number, number, number]
  },
  className: string
}

rotate should be something like "-10deg"
x should be something like [8, 2, 4], it will animate the triangle on the x axis through the values provided in the array. The values given here are random, I let you find the right ones.
y same a x but for the y axis
className will be used to apply a list of css classes

A TrianglesBox component that has no props but display 4 Triangle components.

To position the triangles we can apply a position absolute on them and position relative on parent Item. To animate the triangle component we will have to use the motion api from framer-motion

import { motion } from "framer-motion";

const DummyComponent ({ animateParams }) => (
  <motion.div
    animate={animateParams}
  />
);

We will also need to apply a transition parameter to motion.div to obtain the desired effect.

Ressources:

https://www.framer.com/api/motion/animation/#transitions

Solution:

Triangle component:

https://gist.github.com/flagrede/8143023838936fb4b7cf81eae1c52d54

TrianglesBox component:

https://gist.github.com/flagrede/9587f518c4cf7f8ca991c63a8144b75b

What's next

Congratulations for completing the first part, I hope you enjoyed it !
On part 2 we will implement the pagination between the items categories, the page transition animation and display additional data about the items.
If you encountered any issues or if you have feedback let me know here or on twitter, I will gladly try to address them !

Notes

All the schemas were done using Excalidraw. Thanks to the team for providing such a great tool for free.