How To Embed HTML Into A 3D Model

Into To Blog

This article is assuming that you know how to use the basics of react-three-fiber. So assuming you have a basic layout with react three fiber, lets say a canvas and some 3d model with react three fiber but now you want a 3d model with html embedded into it. This article will be telling you how to exactly do that.

CSS3DRenderer

The approach I used for this was to use CSS3DRenderer from three JS since react three fiber is based on three. CSS3DRenderer renderer is particularly interesting if you want to apply 3D effects to a website without canvas based rendering, But the way we are using it is to have it render dynamically what you want to put in. There are, however, some important limitations like It's not possible to use the material system of three.js and It's also not possible to use geometries. CSS3DRenderer is just focused on ordinary DOM elements. These elements are wrapped into special objects CSS3DObject and then added to the scene graph.

Getting started

First you would want to import all of the things you would need to use CSS3DRenderer.

import { CSS3DObject } from 'three/examples/jsm/renderers/CSS3DRenderer';
import { Canvas as CanvasCSS3D, useThree as useThreeCSS3D, useFrame as CSSFrame } from 'react-three-fiber/css3d';

After importing that you will have to create a constructor that will build out your 3D objects, your construtor should look like this.

function DOMObject({
  dom, position, scale, rotation,
}) {
  const { scene } = useThreeCSS3D();
  const ref = useRef(CSS3DObject);
  CSSFrame(() => {
    ref.current.rotation.y += 0.01;
    ref.current.rotation.x = 0.1;
  })
  useEffect(() => {
    ref.current = new CSS3DObject(dom.current);
    ref.current.position.copy(position);
    ref.current.scale.copy(scale);
    ref.current.rotation.copy(rotation);
    scene.add(ref.current);
    return () => scene.remove(ref.current);
  }, [dom, scene, position, scale, rotation]);
  return null;
}

Now you are all set to create your html embedded 3D models.

Implementation

Now let's head to where we are rendering our canvas and the first thing we will have to do in this component is to wrap the entire canvas in a fragment. Fragments should look like this.

<>
</>

Next thing we have to do is create a reference so the constructor can know what to use to create a 3D model and we can do this by simply adding this.

const ref = useRef(null);

After that you will want to place a new and special canvas under the original canvas but inside your fragments and inside the new canvas you will place your special 3D object constructor so that it’ll render the object.

<>
<Canvas>
      <directionalLight intensity={0.5} />
      <ambientLight intensity={0.5} />
      <spotLight position={[10, 15, 10]} angle={0.9} />
      <Suspense fallback={<Loading />}>
        <Table />
        <Cards />
      </Suspense>
    </Canvas>
    <CanvasCSS3D style={{ position: 'absolute', top: '0' }} camera={{ position: [0, 30, 150] }}>
        <DOMObject
          dom={ref}
          rotation={new THREE.Euler(Math.PI / 4, 0, 0)}
          position={new THREE.Vector3(0, 0, 0)}
          scale={new THREE.Vector3(1, 1, 1)}
        />
      </CanvasCSS3D>
</>

Finally you can create a div right before your fragment and add some html to it and that will render in your canvas as a 3D model but your div must contain the ref that your special 3D object constructor referenced. (ps) you can put an input box or buttons in your div and that will register in the object also.

<div style={{ background: 'green', width: '100px', height: '100px' }} ref={ref}>
        hello
 </div>

Conclusion

If everything is done correctly then you canvas should look like this.