How to display 3D humanoid avatar with React

Tadao Iseki — Sun, 01 Dec 2019 15:28:14 +0000

Introduction

3DCG and VR technology are used in various places and become familiar with us. And the same phenomenon occurs on web browser. Today I introduce VRM and how to display it with React and @pixiv/three-vrm.

What is VRM?

VRM is a file format for handling 3D humanoid avatar (3D model) data for VR applications. If you have an avatar that conforms to VRM, you can enjoy various applications that require a 3D avatar.

What is @pixiv/three-vrm?

pixiv / three-vrm

Use VRM on Three.js

@pixiv/three-vrm is a JavaScript library to use VRM on Three.js. This enables to render VRM on web applications like VRoid Hub.

Prepare VRM

At first, you need to download VRM from VRoid Hub.

Search VRM models by tags.
Select your favorite model.
Move to model page and download by clicking "Use this model"

Set up project

$ npx create-react-app three-vrm-sample
$ cd three-vrm-sample/
$ yarn add @pixiv/three-vrm three react-three-fiber

<!DOCTYPE html>
<html>
  <head>
    <title>@pixiv/three-vrm sample</title>
    <style>
      html,
      body {
        background-color: #000;
        color: #fff;
        margin: 0;
        width: 100vw;
        height: 100vh;
      }

      #root {
        width: 100%;
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>

import React from 'react'
import ReactDOM from 'react-dom'

const App = () => null

ReactDOM.render(<App />, document.getElementById('root'))

Add VRM Loader

We can load VRM with GLTFLoader because VRM is similar format to GLTF.

import { VRM } from '@pixiv/three-vrm'
import { useRef, useState } from 'react'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      setVrm(vrm)
    })
  }

  return { vrm, loadVrm }
}

Display VRM with react-three-fiber

react-three-fiber is a React renderer for Three.js. You can use Three.js declaratively with it. I use the following three elements this time.

<Canvas>: Wrapper element for react-three-fiber elements
<spotLight>: Light element to illuminate objects
<primitive>: 3D object element

When you input a VRM file, handleFileChange() create object url and load VRM.

import React from 'react'
import { Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

Result:

Improve appearance

VRM model in the result is small and facing the other side. You want to see it closer and its face, don't you?

1. Create a new camera from `THREE.PerspectiveCamera` and set its position.

Note:
useThree gives you access to all the basic THREE objects like gl, scene, camera, clock and so on...

import React, { useEffect, useRef } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

2. Rotate and look at camera

Assign camera to vrm.lookAt.target and rotate vrm 180°.

import { VRM } from '@pixiv/three-vrm'
import { useEffect, useRef, useState } from 'react'
import { useThree } from 'react-three-fiber'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}

Final Code:

import { VRM } from '@pixiv/three-vrm'
import ReactDOM from 'react-dom'
import React, { useEffect, useRef, useState } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

ReactDOM.render(<App />, document.getElementById('root'))

Result:

Looks good 👍

Conclusion

VRM will be used more widely in the future. I hope this article will help you when you need to use VRM with React.

@pixiv/three-vrm has more features, so if you are interested, please read the documentation and try it out.

If you have any problems or questions, please write comment or reply to my Twitter account.

Sample Repository:

DEV Community: Tadao Iseki

How to display 3D humanoid avatar with React

Introduction

What is VRM?

What is @pixiv/three-vrm?

pixiv / three-vrm

Use VRM on Three.js

Prepare VRM

Set up project

Add VRM Loader

Display VRM with react-three-fiber

Improve appearance

1. Create a new camera from `THREE.PerspectiveCamera` and set its position.

2. Rotate and look at camera

Conclusion

saitolume / three-vrm-sample

👤Sample repository using @pixiv/three-vrm with React

DEV Community: Tadao Iseki

How to display 3D humanoid avatar with React

Introduction

What is VRM?

What is @pixiv/three-vrm?

pixiv / three-vrm

Use VRM on Three.js

Prepare VRM

Set up project

Add VRM Loader

Display VRM with react-three-fiber

Improve appearance

1. Create a new camera from THREE.PerspectiveCamera and set its position.

2. Rotate and look at camera

Conclusion

saitolume / three-vrm-sample

👤Sample repository using @pixiv/three-vrm with React

1. Create a new camera from `THREE.PerspectiveCamera` and set its position.