From Desktop 3d Apps to Web 3d Apps using Blender and React

In this tutorial, I will walk you thru the steps to create a 3d react application with some interactivity so in the final you will have something like this

• What is Blender? - Simply Explained
• Create React App
• Install dependencies
• Export blender asset
• Compress asset
• Convert asset to JSX component
• Integrate new component
• Enhanced component and functionality
  • Adding some style
  • Install dependency
  • Edit React Components
• Resources
• Appendix

Before you start

You will need to have the following installed or configured and know at least the basics of using them before proceeding.

• NodeJS installed (preferable > 12)
• Basic Knowledge in React
• Previous use of create-react-app
• Not mandatory, but some basic knowledge of using blender 3d app to understand the concept of mesh and material

What is Blender? Simply Explained

This tutorial is not a blender tutorial, so that it will be a short explanation.
Blender is a free, open-source 3D creation suite. With a strong foundation of modeling capabilities, there's also robustly texturing, rigging, animation, lighting, and other tools for complete 3D creation.

Source: Spring - Blender Open Movie Blender, Animation Studio via YouTube

Create React App

npx create-react-app cra-fiber-threejs
npm run start

If everything works successfully, you can navigate to: http://localhost:3000/, and you will see a React App

Install dependencies

• Install gltf-pipeline; this will help you to optimize our glTF, meaning smaller for the web; this is installed globally

npm install -g gltf-pipeline

• Install @react-three dependencies for our project, navigate to cra-fiber-threejs folder and run

npm i @react-three/drei
npm i @react-three/fiber

Export blender asset

• Open blender program with you're created, 3d model
• if you have installed blender and created a 3d modeling, in case you didn't, take a look in the optional step

Optional

• If you have installed blender but didn't create any model, here you have the one I'm using in the tutorial
• If you didn't install blender and want the compressed glb file here, you can download it.

Compress asset

• The file we exported from the previous step, some times are significant, and they are not optimized for the web, so we need to compress it
• Navigate where you saved the .glb file (from the previous step) and run the following command:

gltf-pipeline -i <input file glb> -o <output glb> --draco.compressionLevel=10
e.g:
gltf-pipeline -i shoe.glb -o ShoeModelDraco.glb --draco.compressionLevel=10

Convert asset to JSX component

To start interacting with our 3d model, we need to convert it to a JSX component using gltfjsx. You can read more here. gltfjsx - Turns GLTFs into JSX components)

• Navigate where you saved the .glb file from the previous step and run the following command:

npx gltfjsx <outputed glb from previus step>
e.g. npx gltfjsx ShoeModelDraco.glb

• The output will be a js file with content similar to:

/*
Auto-generated by: https://github.com/pmndrs/gltfjsx
*/

import React, { useRef } from 'react'
import { useGLTF } from '@react-three/drei'

export default function Model({ ...props }) {
  const group = useRef()
  const { nodes, materials } = useGLTF('/ShoeModelDraco.glb')
  return (
    <group ref={group} {...props} dispose={null}>
      <mesh geometry={nodes.shoe.geometry} material={materials.laces} />
      <mesh geometry={nodes.shoe_1.geometry} material={materials.mesh} />
      <mesh geometry={nodes.shoe_2.geometry} material={materials.caps} />
      <mesh geometry={nodes.shoe_3.geometry} material={materials.inner} />
      <mesh geometry={nodes.shoe_4.geometry} material={materials.sole} />
      <mesh geometry={nodes.shoe_5.geometry} material={materials.stripes} />
      <mesh geometry={nodes.shoe_6.geometry} material={materials.band} />
      <mesh geometry={nodes.shoe_7.geometry} material={materials.patch} />
    </group>
  )
}

useGLTF.preload('/ShoeModelDraco.glb')

• The output It's a React component with all the meshed/materials mapped ready to work
• If you worked with blender, you can see that it has mapped all its meshes objects and all its materials
• This component can now be dropped into your scene. It is asynchronous and, therefore, must be wrapped into <Suspense> which gives you complete control over intermediary loading-fallbacks and error handling.

Integrate new component

• Go the project you created using create-react-app
• Copy your new file created in step "Convert asset to JSX component" e.g. ShoeModelDraco.js to src/ folder
• Create a new file for your new component and called it BlenderScene.js, this file will include for the simplicity also some logic and the Scene components, in a real application you will want to separate them in different files/components, copy the following code:

import React, { Suspense } from 'react';
import { Canvas } from "@react-three/fiber"
import { ContactShadows, Environment, OrbitControls } from "@react-three/drei"
import Model from './ShoeModelDraco'
function Scene() {
  return (
    <div className='scene'>
      <Canvas shadows dpr={[1, 2]} camera={{ position: [0, 0, 4], fov: 50 }}>
        <ambientLight intensity={0.3} />
        <spotLight intensity={0.5} angle={0.1} penumbra={1} position={[10, 15, 10]} castShadow />
        <Suspense fallback={null}>
          <Model />
          <Environment preset="city" />
        <ContactShadows rotateX={Math.PI / 2} position={[0, -0.8, 0]} opacity={0.25} width={10} />
        </Suspense>
        <OrbitControls minPolarAngle={Math.PI / 2} maxPolarAngle={Math.PI / 2} enableZoom={false} enablePan={false} />
      </Canvas>
    </div>
  )
}
function BlenderScene() {
  return (
    <>
      <Scene />
    </>

  );
}

export default BlenderScene;

• Copy into the public folder the .glb output file from step "Export blender asset," in my case: ShoeModelDraco.glb

• Use the BlenderScene component you just created, open the App.js file, and import it something like:
  

import './App.css';
import BlenderScene from './BlenderScene';

function App() {
  return (
    <BlenderScene /> 
  );
}

export default App;

• If everything runs successfully, you should see your 3d model in the browser, something like this:

• The only interactivity that you have it's that you can spin the 3d model, and that's it,
• In the following steps, we will:
  • Add more fun/complex interactivity
  • Display nicer in the browser
  • In the resources part, you can find a link for the branch with the code until this step

Enhanced component and functionality

If you are reading here, kudos 💪🏼.
You are almost done 🥵; you have your 3d model in the browser 🎉, but you saw, it's not very interesting and boring; let's start adding cool stuff 😎.

Disclaimer: The following code it's not production-ready, and I did some hacks and also not best practices when writing the components

Adding some style

• Open the App.css file and add in the end of it the following:

#root {
  position: relative;
  margin: 0;
  padding: 0;
  overflow: hidden;
  outline: none;
  width: 100vw;
  height: 100vh;
}
.scene {
    height: 500px;
    padding: 100px;

}

Install dependency

• We will install react-colorful, a tiny color picker component for React and Preact apps. We will use it for choosing colors

npm i react-colorful

Edit React Components

• Open ShoeModelDraco.js file and copy the following code
• We add functionality to work with the mouse when the user clicks on our model
• We add state to know which part of the model was selected

    /*
Auto-generated by: https://github.com/pmndrs/gltfjsx
*/

import React, { useRef } from 'react'
import { useGLTF } from '@react-three/drei'
import { useFrame } from '@react-three/fiber'
export default function Model({ props, currentState, setCurrentState, setHover }) {
  const group = useRef()
  const { nodes, materials } = useGLTF('/ShoeModelDraco.glb');
  // Animate model
  useFrame(() => {
    const t = performance.now() / 1000
    group.current.rotation.z = -0.2 - (1 + Math.sin(t / 1.5)) / 20
    group.current.rotation.x = Math.cos(t / 4) / 8
    group.current.rotation.y = Math.sin(t / 4) / 8
    group.current.position.y = (1 + Math.sin(t / 1.5)) / 10
  })
  return (
    <>
      <group
      ref={group} {...props}
      dispose={null}
      onPointerOver={(e) => {
        e.stopPropagation();
        setHover(e.object.material.name);
      }}
      onPointerOut={(e) => {
        e.intersections.length === 0 && setHover(null);
      }}
      onPointerMissed={() => {
        setCurrentState(null);
      }}
      onClick={(e) => {
        e.stopPropagation();
        setCurrentState(e.object.material.name);
      }}>
      <mesh receiveShadow castShadow geometry={nodes.shoe.geometry} material={materials.laces} material-color={currentState.items.laces} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_1.geometry} material={materials.mesh} material-color={currentState.items.mesh} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_2.geometry} material={materials.caps} material-color={currentState.items.caps} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_3.geometry} material={materials.inner} material-color={currentState.items.inner} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_4.geometry} material={materials.sole} material-color={currentState.items.sole} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_5.geometry} material={materials.stripes} material-color={currentState.items.stripes} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_6.geometry} material={materials.band} material-color={currentState.items.band} />
      <mesh receiveShadow castShadow geometry={nodes.shoe_7.geometry} material={materials.patch} material-color={currentState.items.patch} />
      </group>
    </>
  )
}

useGLTF.preload('/ShoeModelDraco.glb')

• Open BlenderScene.js file and copy the following code
• We add state in order to know which part of the model was selected
• Added work with the picker component
• Added animation to the model, floating illusion

import React, { useState, useEffect, Suspense } from 'react';
import { Canvas } from "@react-three/fiber"
import { ContactShadows, Environment, OrbitControls } from "@react-three/drei"
import { HexColorPicker } from 'react-colorful'
import Model from './ShoeModelDraco'
function Scene() {
  // Cursor showing current color
  const [state, setState] = useState({
    current: null,
    items: {
      laces: "#ffffff",
      mesh: "#ffffff",
      caps: "#ffffff",
      inner: "#ffffff",
      sole: "#ffffff",
      stripes: "#ffffff",
      band: "#ffffff",
      patch: "#ffffff",
    },
  });
  const [hovered, setHover] = useState(null)
  useEffect(() => {
    const cursor = `<svg width="64" height="64" fill="none" xmlns="http://www.w3.org/2000/svg"><g clip-path="url(#clip0)"><path fill="rgba(255, 255, 255, 0.5)" d="M29.5 54C43.031 54 54 43.031 54 29.5S43.031 5 29.5 5 5 15.969 5 29.5 15.969 54 29.5 54z" stroke="#000"/><g filter="url(#filter0_d)"><path d="M29.5 47C39.165 47 47 39.165 47 29.5S39.165 12 29.5 12 12 19.835 12 29.5 19.835 47 29.5 47z" fill="${state.items[hovered]}"/></g><path d="M2 2l11 2.947L4.947 13 2 2z" fill="#000"/><text fill="#000" style="white-space:pre" font-family="Inter var, sans-serif" font-size="10" letter-spacing="-.01em"><tspan x="35" y="63">${hovered}</tspan></text></g><defs><clipPath id="clip0"><path fill="#fff" d="M0 0h64v64H0z"/></clipPath><filter id="filter0_d" x="6" y="8" width="47" height="47" filterUnits="userSpaceOnUse" color-interpolation-filters="sRGB"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feColorMatrix in="SourceAlpha" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 127 0"/><feOffset dy="2"/><feGaussianBlur stdDeviation="3"/><feColorMatrix values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0"/><feBlend in2="BackgroundImageFix" result="effect1_dropShadow"/><feBlend in="SourceGraphic" in2="effect1_dropShadow" result="shape"/></filter></defs></svg>`
    const auto = `<svg width="64" height="64" fill="none" xmlns="http://www.w3.org/2000/svg"><path fill="rgba(255, 255, 255, 0.5)" d="M29.5 54C43.031 54 54 43.031 54 29.5S43.031 5 29.5 5 5 15.969 5 29.5 15.969 54 29.5 54z" stroke="#000"/><path d="M2 2l11 2.947L4.947 13 2 2z" fill="#000"/></svg>`
    if (hovered) {
      document.body.style.cursor = `url('data:image/svg+xml;base64,${btoa(cursor)}'), auto`
      return () => (document.body.style.cursor = `url('data:image/svg+xml;base64,${btoa(auto)}'), auto`)
    }
  }, [hovered])

  function Picker() {
    return (
      <div style={
        {
          display: state.current ? "block" : "none",
          position: "absolute",
          top: "50px",
          left: "50px",

       }
      }>
        <HexColorPicker
          className="picker"
          color={state.items[state.current]}
          onChange={(color) => {
            let items = state.items;
            items[state.current] =  color
          }}
        />
        <h1>{state.current}</h1>
      </div>
    )
  }
  return (
    <div className='scene'>
      <Canvas shadows dpr={[1, 2]} camera={{ position: [0, 0, 4], fov: 50 }}>
        <ambientLight intensity={0.3} />
        <spotLight intensity={0.5} angle={0.1} penumbra={1} position={[10, 15, 10]} castShadow />
        <Suspense fallback={null}>
          <Model
            currentState={ state }
            setCurrentState={(curState) => {
              setState({
                ...state,
                current: curState
              })
            }}
            setHover={ setHover}
          />
          <Environment preset="city" />
        <ContactShadows rotateX={Math.PI / 2} position={[0, -0.8, 0]} opacity={0.25} width={10} />
        </Suspense>
        <OrbitControls minPolarAngle={Math.PI / 2} maxPolarAngle={Math.PI / 2} enableZoom={false} enablePan={false} />
      </Canvas>
      <Picker />
    </div>
  )
}
function BlenderScene() {
  return (
    <>
      <Scene />
    </>

  );
}

export default BlenderScene;

• If everything works successfully, you should see something like this:
  

• In the resources part, you can find a link for the branch with the code until this step

• Live Working example here

Resources

• blender

• gltfjsx

• react-colorful

• STEP_01

• STEP_02

Appendix

• Blender
  • Blender is the free and open-source 3D creation suite. It supports the entirety of the 3D pipeline—modeling, rigging, animation, simulation, rendering, compositing, and motion tracking, even video editing, and game creation; more in here
• glTF files
  • Graphics Language Transmission Format or GL Transmission Format, more here
• gltf-pipeline
  • Content pipeline tools for optimizing glTF, more here