DEV Community: Bryan

Part 9: Asset Handling & Animation (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:51:41 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome back to the 9th and final part of this series about asset handling and animation. In the last article we added a model in form a door to the scene. In the following sections we’re going to implement opening and closing the door on the press of a button.

ℹ️ Remember - you can always run the code associated with this article and follow along using

npm start --part=9

animating the door

Interacting with the door

handleControllerSelection() {
  ...
    this._xr.input.onControllerAddedObservable.add((motionControllerAdded) => {
        motionControllerAdded.onMotionControllerInitObservable.add((motionControllerInit) => {

            ...
            const buttonComponent = motionControllerInit.getComponent(motionControllerComponentIds[3]);

            if (buttonComponent) {
                buttonComponent.onButtonStateChangedObservable.add((component) => {
                    if (component.pressed) {
                        (this._doorIsOpen) ? this.closeDoor() : this.openDoor();
                    }
                });
            }

            triggerComponent.onButtonStateChangedObservable.add((component) => {
            ...
            });
        });
    });
}

First we’re going to add a new interaction to the handleControllerSelectionfunction by observing button presses to the controllers A-Button. The motionControllerComponentIds[3] is the component id related to this specific button.

📚 More about component ids can be found here.

If the button is pressed the door is either opened by the this.openDoor() or closed this.closeDoor() function, depending on the state of this._doorIsOpen.

Opening the door

openDoor() {
  if (this._door !== null) {
      this.animateDoor(30);
      this._doorIsOpen = true;
  }
}

Closing the door

closeDoor() {
  if (this._door !== null) {
      this.animateDoor(30);
      this._doorIsOpen = false;
  }
}

By adjusting the duration we can create an effect where the door opens normally but closes like it is shut. The lower the duration, the quicker the animation is.

Animating the door interaction

animateDoor(duration: number) {

    const animationName = this._doorIsOpen ? "doorOpenQuat" : "doorCloseQuat";
    const doorAnimation = new Animation(animationName, "rotationQuaternion", 30, Animation.ANIMATIONTYPE_QUATERNION, Animation.ANIMATIONLOOPMODE_CONSTANT);

    const startRotation = this._door!.rotationQuaternion!.clone();

    const axis = new Vector3(0, 1, 0); 
    const angle = this._doorIsOpen ? -Math.PI / 1.5 : Math.PI / 1.5; 
    const endRotation = Quaternion.RotationAxis(axis, angle).multiply(startRotation);

    const keyFrames : {frame: number, value: Quaternion}[] = [];

    keyFrames.push({
        frame: 0,
        value: startRotation
    });

    keyFrames.push({
        frame: duration,
        value: endRotation
    });

    doorAnimation.setKeys(keyFrames);

    if (!this._door!.rotationQuaternion) {
        this._door!.rotationQuaternion = new Quaternion();
    }

    this._door!.animations = [doorAnimation];

    this._scene.beginAnimation(this._door, 0, duration, false);
}

To enhance the immersion we implement a smooth animation for the opening and closing the door.

Animation Definition:
- animationName: Determines the name of the animation based on the current state of the door (open or closed).
- doorAnimation: Creates a new Animation object for quaternion rotation. This type of rotation is smooth and avoids issues like gimbal lock. The animation runs at 30 frames per second and doesn't loop.
Initial Rotation:
- Retrieves the door's current rotation in quaternion form (a complex number system that represents rotations).
End Rotation Calculation:
- Defines the rotation axis (y-axis) and the angle (90 degrees for opening, -90 degrees for closing).
- Calculates the end rotation quaternion by multiplying the start rotation by the rotation created from the specified axis and angle.
Keyframe Definition:
- Initializes an array for keyframes, which are crucial points in the animation.
- Adds the start (frame 0) and end (frame at duration) rotations to the keyframes. This defines the animation path from the current rotation to the target rotation.
Animation Assignment:
- Ensures the door is set to use quaternion rotation.
- Assigns the created animation to the door mesh, preparing it to be animated.
Begin Animation:
- Starts the animation using the Babylon.js beginAnimation method on the door mesh from frame 0 to the specified duration.
- The animation is not set to loop.

Conclusion

In this final part of our series, we delved into the nuances of animating a door in a 3D environment, highlighting the seamless integration of user interactions and realistic motion. Utilizing quaternion rotations and Babylon.js, we demonstrated how to create a smooth and responsive door animation that enhances user experience.

Part 8: Models & Assets (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:51:36 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome back to the 8th instalment in this series. This part is about importing an asset in form of a 3D model of a door that gets attached to an anchor.

ℹ️ Remember - you can always run the code associated with this article and follow along using

npm start --part=8

loading and attaching a model to an anchor

Prerequisite

To be able to load model files in form of assets into a scene we first need to import the babylonjs model loader for glb/glTF files.

import '@babylonjs/loaders/glTF';

The provided 3D model file consists of 3 layers. A Handle, a Door and a Door_frame.

The loader itself add a parent node called __root__ to the whole model, therefore the 3 layers are stored as sub meshes to the __root__.

enum DoorMeshes {
    Handle = 'Handle',
    Door = 'Door',
    DoorFrame = 'Door_frame',
    Container = '__root__',
}

To make things easier when loading parts on their own, we create an enum for the layer names.

Loading the model

addDoor(): void {
    SceneLoader.Append("/models/", "door.glb", this._scene, ((scene: Scene) => {

        this._handle = scene.getMeshByName(DoorMeshes.Handle);
        this._door = scene.getMeshByName(DoorMeshes.Door);
        this._doorFrame = scene.getMeshByName(DoorMeshes.DoorFrame);
        this._doorContainer = scene.getMeshByName(DoorMeshes.Container);

        const meshes = this._doorContainer!.getChildMeshes();

        meshes.forEach((mesh) => {
            this._shadowGenerator!.addShadowCaster(mesh);
            mesh.receiveShadows = true;
        });

        this._handle!.isVisible = false;
        this._door!.isVisible = false;
        this._doorFrame!.isVisible = false;
    }));
}

Next we’re going to add the addDoor function to load the 3D model.

SceneLoader.Append loads a model from a specific local file path into the the this._scene.
Once the model is loaded a callback function is called. In the callback function each mesh is assigned to a variable making them easily accessible later on.

❗️The most important layer is the node layer which is assigned to this._doorContainer since this is going to be the one that is being attached to an anchor later on.

Then we loop through all the child meshes to:

add the mesh as a shadow caster to the this._shadowGenerator, allowing it to cast shadows to the scene
setting receiveShadows to true to enable the mesh to receive shadows from other objects in the scene

Finally we set each mesh to be invisible initially to be later explicitly make them visible again when attached to an anchor.

Adding the door to the scene

async createScene(): Promise<Scene> {

  ...
  this.addDoor();

  if (this._xrAnchors && this._xrAnchors.isCompatible()) {
      this.observeAnchors();
      this.handleControllerSelection();
  }

  return this._scene;
}

To be able for the scene to load the model, we add this.addDoor() to the createScene function.

Attaching the model to an anchor

addAnchorAtPosition(raycastHit: PickingInfo) {
    this._xrAnchors!.addAnchorAtPositionAndRotationAsync(raycastHit.pickedPoint!).then((anchor) => {
        const boxTransformNode = new TransformNode('boxTransformNode');

        this._box!.parent = boxTransformNode;
        this._box!.position = new Vector3(0, 1, .5);
        this._box!.isVisible = true;

        this._door!.isVisible = true;
        this._doorFrame!.isVisible = true;
        this._handle!.isVisible = true;

        this._doorContainer!.position = raycastHit.pickedPoint!;
        boxTransformNode.parent = this._doorContainer!;

        anchor.attachedNode = this._doorContainer!;
        anchor.attachedNode.position = raycastHit.pickedPoint!;
    });
}

Since we already know how to attach a mesh to an anchor, we adjust our previous implementation for the this._box to this._doorContainer. Instead of attaching the this._box directly to the anchor, we assign the this._doorContainer as the parent to the box and assign the doorContainer to anchor.attachedNode. Important in this step is to make the model layers visible by setting isVisible to true.

Conclusion

In this 8th installment, we covered the process of importing and integrating a 3D door model with Babylon.js. We detailed loading the model, managing its components, and attaching it to an anchor. The steps taken demonstrate a straightforward approach to incorporating interactive elements in virtual environments, suggesting practical applications in web technology.

In the ninth and final part we’re going to implement some animations for the door.

Part 7: Anchors (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:51:29 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome back to the 7th article in this WebXR/Babylon.js series. This part is about Anchors in WebXR.

ℹ️ Remember - you can always run the code associated with this article and follow along using

npm start --part=7

attaching a mesh to an anchor

What are Anchors?

Anchors are used to create a stable reference point in the physical space, which can be tracked by the device's sensors and cameras. When an anchor is set, the virtual object tied to it appears to be part of the real world, staying in the same place as if it were a physical object.

✅ Anchoring is crucial for maintaining consistency in the virtual environment as it relates to the real world.

Adding anchors to the scene

To add an Anchor to the scene, we go back into our handleControllerSelection, take the raycastHit value and call addAnchorAtPositionAndRotationAsync to add an anchor at the ray cast hit position.

Interacting with the controller

handleControllerSelection() {
  ...
    if (raycastHit.pickedMesh === this._box) {
        const mat = this._box!.material as StandardMaterial;
        mat.diffuseColor = Color3.Random();
        this._box!.material = mat;
    } else {

        this.addAnchorAtPosition(raycastHit);
    }
    this._box!.isVisible = true;
  ...
}

We first check if the mesh we picked is the box and if so, we just give it a randomly different color.

If we don’t pick the box but any other position in the 3D space we will call a new function addAnchorAtPosition with the raycastHit as a parameter.

Adding the Anchor

addAnchorAtPosition(raycastHit: PickingInfo) {
    this._xrAnchors!.addAnchorAtPositionAndRotationAsync(raycastHit.pickedPoint!).then((anchor) => {

        const boxTransformNode = new TransformNode('boxTransformNode');
        boxTransformNode.position = raycastHit.pickedPoint!;

        this._box!.parent = boxTransformNode;
        this._box!.position = Vector3.Zero();
        this._box!.isVisible = true;

        anchor.attachedNode = boxTransformNode;
    });
}

This is where we add a new anchor at our raycastHit.pickedPointposition.

Since anchors have a fixed position in space and don’t support animations, we first have to help ourselves by creating a so called TransformNode for the rotating box. A TransformNode is a versatile and flexible helper that, in our case, acts as a helper to the box by acting as parent to it. Therefore the box keeps it’s rotation animation.

The position of this transform node is set to the point where the ray cast hit, making it align with the XR anchor.

The box is then parented to boxTransformNode. This means any transformation applied to boxTransformNode will also affect the box. The box's position is set to Vector3.Zero(), which means it's placed at the origin relative to its parent (i.e., the boxTransformNode). Since the transform node is already positioned at the ray cast hit point, this effectively places the box there.

this._box!.isVisible = true; makes the box visible in the scene.

Finally, the transform node is attached to the XR anchor. This ensures that the transform node (and therefore the box) will maintain its position relative to the real world, even as the user moves around in the XR space.

Cleaning up

observeAnchors() {
    if (this._xrAnchors === null) {
        return;
    }
    this._xrAnchors.onAnchorAddedObservable.add((addedAnchor) => {
        this._xrAnchors!.anchors.forEach((anchor: IWebXRAnchor) => {
            if (anchor !== addedAnchor) {
                anchor.remove();
            }
        });
    })
}

this._xrAnchors.onAnchorAddedObservable.add(() => { ... }) sets up an observer that will execute the provided callback function every time a new anchor is added to the XR environment.

Within the callback, there's a loop this._xrAnchors!.anchors.forEach((anchor: IWebXRAnchor) => { ... }) that iterates over all the anchors currently in this._xrAnchors.

Inside the loop, anchor.remove() is called on each anchor except the one we just added. This means that every time a new anchor is added to the system, all existing anchors are removed.

The primary intent of this function is to maintain a single active anchor in the XR environment. When a new anchor is added, it triggers the removal of all existing anchors. This is useful in scenarios where only the latest anchor is relevant, and previous anchors should not persist.

Adding `observeAnchors` to the scene

async createScene(): Promise<Scene> {
  ...
  this.observeAnchors();

  return this._scene;
}

To make use of the function we have to add it to createScene.

Conclusion

In conclusion, this seventh installment of our WebXR/Babylon.js series has taken us through the pivotal concept of Anchors in WebXR. Anchors are more than just a technical component in the realm of extended reality; they serve as the vital bridge connecting the virtual and real worlds. By creating these stable reference points in physical space, which remain consistent regardless of the viewer's movement or perspective, we enhance the immersion and believability of our virtual environments.

The practical demonstrations and code examples provided, from handling controller selections to dynamically adding and managing anchors, have illustrated how these concepts are not just theoretical but highly applicable in real-world scenarios. The use of TransformNode as a flexible tool to maintain object animations while anchoring, and the strategic management of anchors to keep the XR environment clean and relevant, show the depth and versatility of Babylon.js in creating compelling XR experiences.

In the eighth part we dive into loading models and assets.

Part 6: Animating a Mesh (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:51:22 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome back to the 6th part of this series about Animations in Babylon.js.

ℹ️ Remember - you can always run the code associated with this article and follow along using

npm start --part=6

animating the box mesh

Essentials & Similarities

Babylon.js offers a robust animation system that allows for the creation and control of animations within 3D scenes.

Animations can be applied to various properties of objects, including position, rotation, scaling, and color.

The system supports keyframe-based animations, where you can define specific values at certain frames, creating smooth transitions over time.

There are similarities to CSS animations:

Keyframe-Based: Both Babylon.js and CSS animations can be keyframe-based, where you define certain states at specific points in time, and the system interpolates the values in between.
Easing Functions: Both can use easing functions to make the transitions between keyframes smooth and more natural.
Transformations: They can both animate transformations like scaling, rotating, and moving objects or elements.
Timeline Control: They offer control over the timing of animations, allowing you to set durations, delays, and iteration counts.

📚 Babylon.js also offers advanced features like blending multiple animations, easing functions for smooth transitions, and the ability to control the playback of animations, including starting, stopping, pausing, and looping.

This flexibility makes it suitable for creating complex and dynamic 3D animations in web applications.

Animating a box

For our experience we’re going to animate a rotation of the box we previously added into our scene.

In effect, this function will rotate the box by 180 degrees around the y-axis, with the rotation occurring smoothly over the span of 100 frames. A looping ensures that the animation will continue indefinitely, creating a rotating box effect in the scene.

These are the steps to accomplish our goal:

animateBox() {
    const rotateAnimation = new Animation("rotateAnimation", "rotation.y", 30, Animation.ANIMATIONTYPE_FLOAT, Animation.ANIMATIONLOOPMODE_CYCLE);

    const keyFrames: { frame: number, value: number }[] = [];

    keyFrames.push({
        frame: 0,
        value: 0
    });

    keyFrames.push({
        frame: 50,
        value: Math.PI / 2
    });

    keyFrames.push({
        frame: 100,
        value: Math.PI
    });

    rotateAnimation.setKeys(keyFrames as IAnimationKey[]);

    this._box!.animations = [rotateAnimation];

    this._scene.beginAnimation(this._box, 0, 100, true);
}

Create Animation Object: A new Animation object named rotateAnimation is created. This animation is set to change the rotation.y property, which affects the y-axis rotation of the box.
Animation Properties:
- 30 specifies the frame rate per second. This indicates how many times the animation values are updated per second.
- Animation.ANIMATIONTYPE_FLOAT indicates that the property being animated (rotation.y) is a float.
- Animation.ANIMATIONLOOPMODE_CYCLE ensures that the animation will repeat in a loop.
Define Key Frames:
- Keyframes are set up to define how the animation progresses over time. Each keyframe specifies a frame number and the corresponding value of the rotation.y at that frame.
- The first keyframe at frame 0 sets the rotation to 0 (start position).
- The second keyframe at frame 50 sets the rotation to Math.PI / 2, which is a 90-degree rotation.
- The third keyframe at frame 100 sets the rotation to Math.PI, a 180-degree rotation.
Apply Keyframes to Animation: The setKeys method of the rotateAnimation object is used to apply the keyframes. This method defines the animation sequence according to the keyframes.
Assign Animation to the Box: The animation array of the _box object (presumably a predefined mesh in the scene) is set to include the rotateAnimation.
Begin Animation: Finally, this._scene.beginAnimation is called with the _box, the start frame (0), the end frame (100), and a boolean to indicate that the animation should loop (true).

Adding the animation to the scene

async createScene(): Promise<Scene> {
  ...
  this.animateBox();

  return this._scene;
}

Finally, we’re adding the this.animateBox() function to the scene.

Conclusion

This article demonstrates the powerful and flexible animation system of Babylon.js, ideal for creating dynamic 3D animations in web applications. Through a practical example of animating a box, it showcases how to set up keyframe-based animations, define keyframes, and apply them to create smooth and looping animations, emphasizing Babylon.js's capability to bring 3D objects to life with ease and precision.

The seventh part of the series will be about Anchors.

Part 5: Input/Controllers & Ray Casting (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:51:16 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome back to the 5th installment of this WebXR/Babylon.js series. This part is about Input (specifically MetaQuests Motion Controllers) and the concept of Ray Casting.

ℹ️ Remember - you can always run the code associated with this article and follow along using

npm start --part=5

interacting with a mesh by changing its color

Ray Casting

What is a Ray Cast?

Ray casting is used to determine the line of sight, select or interact with virtual objects, or navigate through a virtual environment.

How does it work?

Ray casting involves projecting a line (ray) from a specific point into the virtual environment and determining what objects in the virtual world it intersects with. For example, in a VR game, you might use ray casting to select an item or to aim a weapon.

Hit Testing vs. Ray Casting

❗️ Hit testing and ray casting might be perceived as similar because both involve projecting rays in a 3D environment to determine intersections. This fundamental similarity in their core concept—using rays to find points of interaction—can make them seem alike, especially in scenarios where both methods are used for spatial analysis or object interaction within virtual or augmented reality environments.

✅ Hit testing in augmented and virtual reality determines where a virtual object intersects with the real-world environment, while ray casting involves projecting a line from a point in space to detect intersections primarily within a virtual environment.

Performing a Ray Cast from a Controller Input Source

createRayFromController(controller: WebXRInputSource): Ray {
    const origin = controller.pointer.position;
    const direction = controller.pointer.forward;
    return new Ray(origin, direction, length = 100);
}

The createRayFromController method in this context generates a Ray object originating from a WebXR controller. It takes a WebXRInputSource object (representing the controller) as an argument. The method sets the ray's origin to the controller's pointer position and its direction to the forward direction of the controller's pointer. The ray is given a length of 100 units. This functionality is typically used in virtual environments to create a directional line for purposes like aiming or selecting objects in the direction the controller is pointing.

Controller Detection and Interaction

📚 Babylon.js supports a wide range of motion controllers. Each controller's buttons, thumb sticks, and touchpads are updated every frame, providing real-time input data.

handleControllerSelection() {

    if (this._xr === null) {
        return;
    }
    this._xr.input.onControllerAddedObservable.add((motionControllerAdded) => {
        motionControllerAdded.onMotionControllerInitObservable.add((motionControllerInit) => {
            const motionControllerComponentIds = motionControllerInit.getComponentIds();
            const triggerComponent = motionControllerInit.getComponent(motionControllerComponentIds[0]);

            triggerComponent.onButtonStateChangedObservable.add((component) => {
                if (component.pressed && component.value > 0.8) {
                    const resultRay = this.createRayFromController(motionControllerAdded);
                    const raycastHit = this._scene.pickWithRay(resultRay);
                    if (this._debug) console.log(raycastHit);

                    if (raycastHit && raycastHit.hit && raycastHit.pickedMesh) {
                        if (raycastHit.pickedMesh === this._box) {
                            const mat = this._box!.material as StandardMaterial;
                            mat.diffuseColor = Color3.Random();
                            this._box!.material = mat;
                        }
                    }
                }
            });
        });
    });
}

The function handleControllerSelection() illustrates how to respond to inputs from a motion controller in an XR scene.

Initialization: First, it checks if the XR session is present and initialized.

Controller Events: When a controller is added, an observable onControllerAddedObservable is activated. This observable responds to the controller's initialization by listening to the controller’s onMotionControllerInitObservable.

Trigger Component: Once the controller is initialized, the trigger component of the controller (usually the trigger button) is identified. This component is then used to respond to press events by listening to the onButtonStateChangedObservable.

motionControllerComponentIds[0] corresponds to the standard trigger button on an XR controller like a MetaQuest 3 controller.

The observer reacts to us pressing the trigger button on the controller. Since the trigger button can be pressed to various degrees, we want to make sure the button is at least pressed for 80%, so quite strong, before reacting to it. If both requirements are met, the ray cast is created from the controller.

If our ray cast hits the box (if we are aiming at it and pulling the trigger on our controller), the color of the box will randomly change to a different color.

More about inputs and controllers can be found here

Adding the Controller Selection to the scene

async createScene(): Promise<Scene> {
  ...
  this.handleControllerSelection();

  return this._scene;
}

Finally, we add the handleControllerSelection to the scene.

Conclusion

Babylon.js provides robust support for various input sources and controllers in WebXR, effectively managing them through classes like WebXRInput and WebXRInputSource. This flexibility allows for seamless integration and real-time interaction with different types of controllers, enhancing the user experience in virtual environments.

This article demystifies the concepts and practical applications of ray casting and hit testing in virtual environments, illustrating their similarities and differences. It emphasizes the use of ray casting for interactions within virtual spaces, such as selecting or aiming at objects, and demonstrates how to implement ray casting in a practical scenario using WebXR controllers. This highlights the importance and versatility of these techniques in developing immersive and interactive virtual reality experiences.

In the sixth part we dive into Animations.

Part 4: Hit Testing (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:51:10 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome back to the next part of our series. This time we learn about Hit Testing.

ℹ️ Remember - you can always run the code associated with this article and follow along using

npm start --part=4

a torus making a hit test visible

What is a Hit Test?

ℹ️ A hit test in WebXR is typically used to determine where in the real world a virtual object should be placed or how it should interact with the real environment.

📚 We will achieve this by creating a torus mesh (looks like a Donut 🍩) that’ll show us on which planes a hit can be detected and where virtual objects can be placed in accordance to the physical environment. Like placing an object on the floor, or on a detected table.

How it works

A hit test involves sending a virtual ray from a specific point (often the user's viewpoint or a point in virtual space) and checking where this ray intersects with real-world surfaces.

📚 While it is common to use the camera (representing the user's view in most cases) as the origin for hit tests, especially in scenarios like first-person views or camera-based selections, Babylon.js is flexible enough to allow hit tests to be initiated from any point in the 3D space.

Use cases

Camera-Based Hit Testing: The most straightforward and common scenario, where the hit test is performed from the camera's viewpoint. This is typically used for selecting or interacting with objects in the scene based on the user's current view.

Arbitrary Point in Space: Developers can initiate a hit test from any arbitrary point in the 3D space, not just the camera. This is useful for scenarios where you need to check for intersections or collisions at specific locations in the scene, independent of the camera's position.

Moving Objects: Hit tests can also be initiated from moving objects within the scene, like a character's hand or a moving vehicle, to check for interactions or collisions with other objects in the scene.

📚 The choice of the origin for a hit test depends on the specific requirements of the application being developed.

ℹ️ Hit testing often relies on the detection of planes or surfaces in the real world to place virtual objects accurately.

Adding a Mesh as a marker

First we create a Mesh in form of a torus that serves us as a visual marker for a hit test.

addMarkerForHitTest() {
    this._marker = MeshBuilder.CreateTorus("marker", { diameter: 0.15, thickness: 0.05 }, this._scene);
    this._marker.isVisible = false;
    this._marker.receiveShadows = true;
    this._marker.checkCollisions = true;
    this._marker.rotationQuaternion = new Quaternion();
}

Performing a Hit Test

Next we perform hit testing to determine the interaction between the real world and virtual objects.

addFeaturesToSession() {
  ...
  try {
    ...
    this._xrHitTest = this._fm.enableFeature(WebXRFeatureName.HIT_TEST, "latest") as WebXRHitTest;
  } catch (error) {
    ...
  }
  ...
}

First we initialize and enable the hit testing feature in Babylon.js by adding it as a feature to our session.

performHitTest() {
    if (this._xrHitTest === null || this._marker === null) {
        return;
    }
    this._xrHitTest.onHitTestResultObservable.add((results) => {
        if (results.length) {
            this._marker!.isVisible = true;
            this._hitTest = results[0];
            this._hitTest.transformationMatrix.decompose(undefined, this._marker!.rotationQuaternion!, this._marker!.position);
        } else {
            this._marker!.isVisible = false;
            this._hitTest = undefined;
        }
    });
}

As soon as hit test results are available, the onHitTestResultObservable.add observer method is called.

When hit test results are present (results.length > 0):

Makes the marker visible.
Stores the first hit test result (results[0]) in a local variable (_hitTest).
Uses the transformation matrix of the hit test result to update the position and orientation of the marker in the scene. This positions and aligns the marker at the point where the hit test detected a collision with a real surface.

If no hit test results are present, the marker is made invisible again and the local variable _hitTest is reset.

Adding Hit Testing to the scene

async createScene(): Promise<Scene> {
  ...
  this.addMarkerForHitTest();
  this.performHitTest();

  return this._scene;
}

Once again, we have to add the addMarkerForHitTestand the performHitTest functions to the scene.

Conclusion

In summary, hit testing in WebXR, particularly with Babylon.js, is an essential tool for creating interactive augmented reality experiences. It allows developers to place virtual objects accurately within the real world, enhancing the realism and immersion of XR applications. The ability to initiate hit tests from different points, like the user’s camera or specific locations in 3D space, provides versatility in application design. Understanding and applying hit testing is key to develop engaging and believable content.

In the fifth part of this series we take a look at input controllers and ray casting.

Part 3: Meshes & Materials (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:51:04 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome back to the next article in our WebXR series. In this article, we take a look at some helpful features implemented in Babylon.js, specifically related to creating objects.

ℹ️ Remember - you can always run the code associated with this article and follow along.

npm start --part=3

Creating a box mesh

What is Material?

ℹ️ Materials give your meshes color and texture.

They can be displayed as a wireframe, scaled and offset across a mesh, have degrees of transparency and be blended.

What is a Mesh?

ℹ️ In the 3D virtual world shapes are built from meshes, lots of triangular facets joined together, each facet made from three vertices.

a mesh

Babylon.js splits meshes into 4 broader categories.

Set Shapes - which usually have names in everyday use and their forms are well known and recognized. Examples include the box, sphere, cone and plane.

Set Shapes

Parametric Shapes - these have no everyday names and are formed through data sets and tend to have irregular shapes. Examples include ribbons, extrusions, and irregular convex or concave polygons.

Parametric Shapes

Polyhedra - three dimensional regular solids including octahedron, dodecahedron, icosahedron and icosphere. The method of creating these is createPolyhedron along with a number for the basic shapes and an array of vertices for a wider range of shapes or createIcoSphere.

Polyhedra

Custom - those you design yourself and build from your own set of vertices connected into triangular facets as you choose.

📚 More about Meshes can be found here.

✅ For the simplicity of this series we’re going to create a simple Box with the help of the Meshbuilder and give it a color using a StandardMaterial.

Creating a Box Mesh

The function createBox() illustrates how easily a 3D object can be created in Babylon.js.
In this example, a simple box is created.

createBox() {
    const material = new StandardMaterial("material", this._scene);

    material.diffuseColor = Color3.Random();

    this._box = MeshBuilder.CreateBox("box", { width: 0.5, height: 0.5, depth: 0.5 }, this._scene);
    this._box.material = material;
    this._box.rotation.y = Math.PI / 4;
    this._box.position = new Vector3(0, 1.5, 1.5);
}

First, a standard material is created, calling StandardMaterial("material", this._scene). Then, the material's diffuse color is set to a random color by calling Color3.Random().

The box itself is created with MeshBuilder.CreateBox("box", { width: 0.5, height: 0.5, depth: 0.5 }, this._scene). This method creates a cube-shaped mesh with the specified dimensions.

The box is then rotated by 45 degrees Math.PI / 4 and positioned at (-1, 1.5, 1.5) in 3D space.
This will place the box a little to the front left relative to us.

Adding the box to the scene

async createScene(): Promise<Scene> {
  ...
  this.createBox();

  return this._scene;
}

Finally, we have to add this.createBox() to the scene.

Conclusion

In wrapping up this article, we've touched on the fundamentals of creating 3D objects in Babylon.js within the WebXR framework. Through the simple box mesh example, we've seen how materials and meshes work together in 3D modeling.

The fourth part of this series will be about Hit Testing.

Part 2: Plane Detection (WebXR with Babylon.js)

Bryan — Thu, 11 Jan 2024 08:50:56 +0000

👀 Stumbled here on accident? Start with the first part!

Welcome to the second part to our WebXR journey. We previously setup our development environment and went through our base template.

In this part we dive into the plane detection feature.

ℹ️ Remember - you can always run the code associated with this article and follow along using

npm start --part=2

What plane detection looks like

What is plane detection?

📚 Plane detection in WebXR identifies and maps flat surfaces in the user's environment for augmented reality applications. This allows virtual objects to be realistically placed on floors, walls, or tables. It enhances AR experiences by ensuring seamless integration of virtual content with the physical world.

Prerequisites

❗️ For this part we assume that you went through the Assisted Space Setup on the Meta Quest 3. The assistant can be found under Settings->Physical Space->Space Setup

Settings → Physical Space

Space Setup (left) → Space Setup (right)

Assisted Space Setup

The Assisted Space Setup feature on the Meta Quest 3 enhances the virtual reality experience by allowing the headset to interact with your physical environment.

3D Environmental Scanning: When activated, the device performs a quick 3D scan of your surroundings.
Object Recognition: It identifies and creates representations of surfaces and objects, like walls, tables, and furniture.
Spatial Interaction: This allows virtual content to collide with or hide behind these real-world objects, making the XR experience more immersive.
Automatic Activation: The feature runs automatically when you launch apps that use mixed reality features, but it can also be accessed manually from the settings.

The Assisted Space Setup is especially useful in mixed reality applications where interaction with the physical environment is key to the experience.

Registering the plane detection feature

Feature Management

The Plane Detection feature is enabled via the WebXRFeaturesManager. This is done by calling enableFeature(WebXRFeatureName.PLANE_DETECTION, "latest").

addFeaturesToSession() {
    if (this._xr === null) {
        return;
    }

    this._fm = this._xr.baseExperience.featuresManager;

    try {
        this._xrPlanes = this._fm.enableFeature(WebXRFeatureName.PLANE_DETECTION, "latest") as WebXRPlaneDetector;
    } catch (error) {
        console.log(error);
    }
}

Performing the Plane Detection

Plane detection uses the WebXR API to recognise real-world surfaces. By accessing the camera data and sensors of the device, it can identify different types of planes, such as horizontal and vertical surfaces.

Events such as onPlaneAddedObservable, onPlaneUpdatedObservable, and onPlaneRemovedObservable are used to respond to changes in the detected plane landscape. These events control the creation, updating, and removal of the meshes that represent the physical planes.

this._xrPlanes.onPlaneAddedObservable.add((plane: IWebXRPlaneWithMesh) => {
    mat = new StandardMaterial("mat", this._scene);
    mat.alpha = 0.25;
    mat.diffuseColor = Color3.Random();
    this.initPolygon(plane, mat);
});

onPlaneAddedObservable: When a new plane is detected, this observable adds an event listener. A new StandardMaterial is created with some level of transparency (alpha = 0.25) and a random diffuse color. The initPolygon function is then called to create a mesh for this plane.

this._xrPlanes.onPlaneUpdatedObservable.add((plane: IWebXRPlaneWithMesh) => {
    if (this._planes[plane.id].material) {
        mat = this._planes[plane.id].material as StandardMaterial;
        this._planes[plane.id].dispose(false, false);
    }
    const some = plane.polygonDefinition.some(p => !p);
    if (some) {
        return;
    }
    this.initPolygon(plane, mat!);
});

onPlaneUpdatedObservable: This listens for updates to existing planes. If the plane's mesh already has a material, it retrieves and reuses this material; otherwise, it disposes of the current mesh and calls initPolygon to recreate it. This ensures the mesh is always up-to-date with the latest plane data.

this._xrPlanes.onPlaneRemovedObservable.add((plane: IWebXRPlaneWithMesh) => {
    if (plane && this._planes[plane.id]) {
        this._planes[plane.id].dispose()
    }
})

onPlaneRemovedObservable: It listens for when a plane is no longer detected and disposes of the corresponding mesh to free up resources.

if (this._xr !== null) {
    this._xr.baseExperience.sessionManager.onXRSessionInit.add(() => {
        this._planes.forEach((plane: Mesh) => plane.dispose());
        while (this._planes.pop());
    });
}

The code checks if the _xr object (representing the XR experience) is not null and adds an event listener for the XR session's initialization. This listener disposes of all plane meshes, effectively resetting the plane representations when a new XR session starts.

The complete code

createPlaneMeshesFromXrPlane(): void {

  interface IWebXRPlaneWithMesh extends IWebXRPlane {
      mesh?: Mesh;
  }

  let mat: Nullable<StandardMaterial>;

  if (this._xrPlanes === null) {
      return;
  }

  this._xrPlanes.onPlaneAddedObservable.add((plane: IWebXRPlaneWithMesh) => {
      this._debug && console.log("plane added", plane);
      mat = new StandardMaterial("mat", this._scene);
      mat.alpha = 0.25;
      mat.diffuseColor = Color3.Random();
      this.initPolygon(plane, mat);
  });

  this._xrPlanes.onPlaneUpdatedObservable.add((plane: IWebXRPlaneWithMesh) => {
      if (this._planes[plane.id].material) {
          mat = this._planes[plane.id].material as StandardMaterial;
          this._planes[plane.id].dispose(false, false);
      }
      const some = plane.polygonDefinition.some(p => !p);
      if (some) {
          return;
      }
      this.initPolygon(plane, mat!);
  });

  this._xrPlanes.onPlaneRemovedObservable.add((plane: IWebXRPlaneWithMesh) => {
      if (plane && this._planes[plane.id]) {
          this._planes[plane.id].dispose()
      }
  })

  if (this._xr !== null) {
      this._xr.baseExperience.sessionManager.onXRSessionInit.add(() => {
          this._planes.forEach((plane: Mesh) => plane.dispose());
          while (this._planes.pop());
      });
  }}
}

Mesh Visualisation

For each detected plane, a mesh is created that provides a visual representation of the plane in the virtual world. These meshes are equipped with materials created using StandardMaterial and are coloured with Color3.Random() for visual distinction.

initPolygon(plane: IWebXRPlane, material?: StandardMaterial): Mesh {}

Initializing the function by providing a plane and a material.

plane.polygonDefinition.push(plane.polygonDefinition[0]);

Adds the first point of the polygon definition to the end, making it a closed polygon.

const polygonTriangulation = new PolygonMeshBuilder(plane.xrPlane.orientation, plane.polygonDefinition.map((p) => new Vector2(p.x, p.z)), this._scene);

Create a new PolygonMeshBuilder object, using the plane.polygonDefinition to define the shape of the polygon. The map function is used to convert each point in plane.polygonDefinition to a Vector2 object, using the x and z properties of each point.

const polygon = polygonTriangulation.build(false, 0.01);

Build the polygon mesh using the PolygonMeshBuilder object. The false argument means that the mesh is not updatable. The 0.01 argument is the depth of the mesh.

polygon.createNormals(false);

Creating the normals for the polygon. Normals are vectors perpendicular to the surface of the mesh, used for lighting calculations. The false argument means that the normals are not updated.

if (material) {
  polygon.material = material;
}

Assign a material to the polygon, if one is provided. A material defines the appearance of the mesh.

polygon.rotationQuaternion = new Quaternion();

Initialize the rotation of the polygon using a quaternion. Quaternions are a way to represent rotations in 3D space.

polygon.checkCollisions = true;
polygon.receiveShadows = true;

Enable collisions and shadows for the polygon.

plane.transformationMatrix.decompose(polygon.scaling, polygon.rotationQuaternion, polygon.position);

Decompose the transformation matrix of the plane into scaling, rotation, and position components, and apply them to the polygon.

this._planes[plane.id] = (polygon);

Adds the polygon to the planes array, using the plane id as the key.

return polygon;

Returns the polygon

The complete code

initPolygon(plane: IWebXRPlane, mat?: StandardMaterial): Mesh {
    plane.polygonDefinition.push(plane.polygonDefinition[0]);

    const polygonTriangulation = new PolygonMeshBuilder(plane.xrPlane.orientation, plane.polygonDefinition.map((p) => new Vector2(p.x, p.z)), this._scene);
    const polygon = polygonTriangulation.build(false, 0.01);

    polygon.createNormals(false);

    if (mat) {
        polygon.material = mat;
    }

    polygon.rotationQuaternion = new Quaternion();

    polygon.checkCollisions = true;
    polygon.receiveShadows = true;

    plane.transformationMatrix.decompose(polygon.scaling, polygon.rotationQuaternion, polygon.position);

    this._planes[plane.id] = (polygon);

    return polygon;
}

Adding Plane Detection to the scene

async createScene(): Promise<Scene> {
  ...
  this.createPlaneMeshesFromXrPlane();

  return this._scene;
}

Finally we have to add our createPlaneMeshesFromXrPlane function to the scene.

Conclusion

This article details the implementation of plane detection in WebXR, specifically focusing on its integration within the Meta Quest 3's Assisted Space Setup. It describes how the WebXR API is utilized to identify real-world surfaces, allowing for the placement of virtual objects in an augmented reality environment. Key features include the creation, updating, and removal of meshes representing detected planes, enhancing the realism and interactivity of the mixed reality experience.

In the third part of this series we’re focusing on Meshes & Materials. Stay tuned.

Welcome to the Exciting World of WebXR and Babylon.js

Bryan — Thu, 11 Jan 2024 08:50:36 +0000

This article series serves as a first step into WebXR development using Babylon.js, a powerful and user-friendly framework. This journey will help you make your first steps into XR on the web.

ℹ️ The goal of this series is for you to find yourself immersed in an interactive mixed reality scene. Each article is building up on each other. You’ll tackle every aspect needed, to create the final scene.

The goal of this series

Let’s dive in!

What Is WebXR?

WebXR is a technology that allows web developers to create immersive 3D XR experiences directly in web browsers. It's a game-changer for the field of virtual and augmented reality, making these experiences more accessible than ever before.

Browser support

According to caniuse WebXR is still widely unsupported and if supported, only partially. Safest bet currently is using either Chrome or a Chromium based browser (Edge, Arc, Opera)

Why Babylon.js?

Babylon.js stands out as a robust framework for building 3D XR experiences. Its ease of use and powerful features make it an ideal choice for developers venturing into the world of WebXR.

Your Learning Journey

📚 The best way to go through each article is by starting the related code part and either viewing it in the browser or directly on the Meta Quest 3.

In this series, we'll start with the fundamentals of WebXR programming and gradually move to more advanced techniques. Each article will provide practical guidance and insights into AR development. You'll learn how to:

Set up your development environment.
Understand and use Babylon.js core features like:
- Plane detection
- Meshes & Materials
- Hit-Testing
- Input & Controllers & Ray Casting
- Animation
- Anchors
- Models & Assets
Create an immersive and interactive 3D scene.

Prerequisites

ℹ️ Before diving in, make sure you have

Node und npm installed (v20.9.0 - 11/2023)
A basic understanding of Javascript and Typescript
A MetaQuest 3 Headset

📚 Because of the nature of Babylon.js being a game engine, we will come across terms like vertices, polygons and meshes as well as some geometry. So a little background on these topics will be helpful but not needed.

Setting Up Your Environment

Here's a step-by-step guide to get your development environment ready:

ℹ️ The base project can be found here: Github/webxr-article-series

Download the Base Project: Start by downloading our base project specifically designed for WebXR and Babylon.js. This will give you a base project to work with.
Install Dependencies: Open your terminal, navigate to your project's directory, and run npm install. This command will download and install all the necessary dependencies for your project.
Start Your Webpack Server: Run npm start to fire up your Webpack server. Webpack is a module bundler that will help in efficiently compiling and running your code.
This will load the first part of code for this article.
To load any other part run npm start --part=[1-10].
Accessing Your Project on XR Device: Use LocalTunnel to expose your local web server to the internet. This will allow your XR device to access your project. Simply run npx localtunnel --port 8080, replacing 8080 with your server's port number (Webpack uses 8080 per default).

ℹ️ LocalTunnel can be very slow at times. An alternative is TunnelMole.

Testing Your Setup

❗️ The first time you access the site provided by LocalTunnel, you will land on a page similar to the following. For security reasons, we have to enter our IP address in order to be able to access our local web server. You can view your IP at https://ipv4.icanhazip.com/. Copy this into the input field for Endpoint IP and confirm by clicking on the button.

LocalTunnel setup page

Once everything is set up, you should be able to access your project through a public URL provided by LocalTunnel. Open the link provided by LocalTunnel (shown in your terminal) in a browser of your choice.

❗️ The page load can be slow through LocalTunnel. The scene will always load eventually.

Test this link on your MetaQuest 3 headset or any other AR-enabled device to ensure everything is working correctly. You should see a page similar to the one below. To enter the XR scene, simply click on the button on the bottom right.

📚 To exit an XR session in the Meta Quest 3 headset, you have to press the A or X button on either of the controllers and choose Quit to end the session. You’ll be going back to your browser window.

This is also needed every time you make any code changes or when re-/starting Webpack.

The immersive web emulator for the browser

📚 If you don't have a device at hand, you can install Meta's Immersive Web Emulator as a browser extension for Chrome and Chromium-based browsers.

Immersive Web Emulator

The WebXR emulator interface

The base template

ℹ️ The base template will always be your point of reference for the whole journey. Each part of the series will build up on the prior and will finish with a link to the then current state of our project.

You can find the different parts called index_[part].ts under the src folder

The base project structure

Let’s begin by going through the base template step by step. We will start seeing actual progress in the scenes composition in the next article. This part is to make you familiar with the base construct creating an XR scene in Babylon.js.

Importing the Babylonjs dependencies and the debugger.

import {
    Engine,
    Scene,
    Vector3,
    WebXRDefaultExperience,
    ShadowGenerator,
    DirectionalLight,
    IShadowLight,
    HemisphericLight,
} from '@babylonjs/core';

import { Inspector } from '@babylonjs/inspector';

Defining possible session modes for WebXR.

type SessionModes = "immersive-ar" | "immersive-vr" | "inline";

These modes dictate how the content will be presented to the user and what kind of input and output capabilities are available. The primary session modes in WebXR are:

Immersive VR: This mode is designed for fully immersive VR experiences. When a session is started in immersive VR mode, it takes over the entire display, blocking out the real world entirely. This mode is typically used with VR headsets, providing a full 360-degree view of a virtual environment. It's ideal for games, simulations, and other applications where complete immersion in a virtual world is desired.
Immersive AR: This mode is for augmented reality experiences. Unlike immersive VR, immersive AR allows the user to see the real world around them, with digital content superimposed. This mode is used with AR-capable devices, like some smartphones and dedicated AR headsets. It's suitable for applications that augment the real world with additional information or virtual objects, such as navigation aids, information overlays, or interactive games that integrate with the physical environment.
Inline: This mode is for non-immersive experiences that are displayed within a standard web page, typically in a canvas element. The inline session mode doesn’t take over the full display and is not exclusive to AR or VR. It's useful for 3D rendering that integrates with other web content, like product previews or simple interactive 3D models. You don't need any special glasses or headsets to view it. It's like seeing a 3D model or interactive scene appear on a regular webpage.

Defining possible reference space types for WebXR.

These reference space types are crucial for creating immersive and interactive 3D experiences in a virtual environment. The main reference space types in WebXR are:

Reference Space Type	Origin Position	Movement Range	Ideal Use Case
Local	Near users initial position	Limited to initial position	Seated or standing experiences with minimal movement
local-floor	Floor level	Limited to initial position	Interactions at floor level, such as games or training simulations
bounded-floor	Floor level	Within a defined area (e.g. a room)	Room-scale VR with more physical movement and interaction
unbounded	Not fixed (adjusts with user)	Unlimited, no predefined boundaries	Large-scale AR with free movement in the real world
viewer	Users current position	Relative to users movement	Head-tracking and rendering user perspective

Creating our main class and variables

class XrExperience {
    // The canvas element is used to display the scene.
    _canvas: HTMLCanvasElement;

    // The 3D engine is responsible for creating and updating the scene.
    _engine: Engine;

    // The scene is the 3D environment where the 3D models are displayed.
    _scene: Scene;

    // debug mode
    _debug: boolean;

    // xrExperience helper
    _xr: WebXRDefaultExperience | null;

    // xrExperience options
    _sessionMode: SessionModes;
    _referenceSpaceType: ReferenceSpaceType;
    _optionalFeatures: boolean;

The constructor is setting up the necessary elements for a 3D scene and configuring the WebXR experience.

Error Handling for WebGL Support

if (!Engine.isSupported()) {
    throw 'WebGL not supported';
}

This segment checks if WebGL is supported in the user's browser. WebGL (Web Graphics Library) is essential for rendering 3D graphics in web browsers. If WebGL is not supported (Engine.isSupported() returns false), the constructor throws an error, stopping further execution. This is important for ensuring that the application only runs in environments where it can function correctly.

Setting Up the Canvas and Engine

this._canvas = document.getElementById('canvas') as HTMLCanvasElement;
this._engine = new Engine(this._canvas, true);

Here, the constructor locates a canvas element in the DOM (Document Object Model) with the ID renderCanvas and assigns it to this._canvas. The canvas is where the 3D scene will be rendered. Then, it creates a new instance of the Babylon.js Engine, passing the canvas as an argument. This engine is responsible for the rendering and overall management of the 3D scene.

Creating the Scene

this._scene = new Scene(this._engine);

This line initializes a new Scene object using the engine. The scene is where all the 3D objects, lights, cameras, and other elements will be added.

Debugging and XR Configuration

this._debug = true;
this._sessionMode = "immersive-ar";
this._referenceSpaceType = "local-floor";
this._optionalFeatures = true;

The constructor sets up some properties for debugging and configuring the WebXR experience. It sets the session mode to immersive-ar (augmented reality) and the reference space type to local-floor, indicating the type of XR experience and the reference space for tracking.

Initializing the XR Experience

this._xr = null;
this.createXrExperience().then(() => {
    this.createScene().then(() => {
        ...
    });
}).catch((error) => {
    console.log(error);
});

These lines initialize the XR experience and the scene creation. The createXrExperience and createScene methods are custom methods in the class that set up the XR experience and populate the scene with content.

Render Loop and Resize Event Handling

this._engine.runRenderLoop(() => {
    this._scene.render();
});
window.addEventListener('resize', () => {
    this._engine.resize();
});

The runRenderLoop method of the engine is called with a function that renders the scene. This creates an infinite loop that continuously renders the scene, allowing for animations and real-time updates. Additionally, an event listener is added to the window object to handle resize events, ensuring that the engine adjusts its rendering to fit the new window size.

Creating a scene

This creates the scene and sets up lights and shadows. This is also the point where the debug inspector can be enabled.

async createScene(): Promise<Scene> {

  this.createLightsAndShadows();

  if (this._debug) Inspector.Show(this._scene, {}); // enable for debugging

  return this._scene;
}

Creating Lights and Shadows

First 2 different types of lights are created to support the scene.

createLightsAndShadows() {
  const lights = this.createLights();

  const shadowGenerator = new ShadowGenerator(1024, lights as IShadowLight);

  shadowGenerator.useBlurExponentialShadowMap = true;
  shadowGenerator.blurKernel = 32;
}

createLights() {

  const directionalLight = new DirectionalLight("directionalLight", new Vector3(0, 1, 1), this._scene);
  directionalLight.intensity = 0.7;
  directionalLight.position = new Vector3(0, 2, 0);

  const hemiLight = new HemisphericLight("hemisphericLight", new Vector3(0, 1, 0), this._scene);
  hemiLight.intensity = 0.7;

  return directionalLight;
}

Directional Light: A DirectionalLight with a specified direction (new Vector3(0, 1, 1)), indicating the direction the light is coming from. Its intensity is set to 0.7, and it's positioned at new Vector3(0, 2, 0) in the scene.
Hemispheric Light: A HemisphericLight is also created. This type of light emits light from the direction of the specified vector (new Vector3(0, 1, 0)) and lights the scene with a gradient from the sky (light) to the ground (dark). Its intensity is similarly set to 0.7.

Then a ShadowGenerator is instantiated to create a shadow with a map size of 1024. The size of the shadow map affects the quality and performance of the shadow rendering.

The shadow generator's properties are set to use a blur exponential shadow map (useBlurExponentialShadowMap = true) and a blur kernel size of 32. These settings enhance the visual quality of the shadows, making them softer and more realistic by reducing artefacts like aliasing (with the help of exponential shadow mapping) and blurring to further smoothing the shadows.

Starting point of our application.

new XrExperience();

Conclusion

This concludes the first part of our journey into WebXR and Babylon.js, offering a solid foundation in developing AR experiences. This sets the stage for continued learning and development in this dynamic field, inviting you to stay tuned for further insights and enhancements in web-based mixed reality.

The second part of our journey will be about Planes in WebXR.

DEV Community: Bryan

Part 9: Asset Handling & Animation (WebXR with Babylon.js)

Interacting with the door

Opening the door

Closing the door

Animating the door interaction

Conclusion

Part 8: Models & Assets (WebXR with Babylon.js)

Prerequisite

Loading the model

Adding the door to the scene

Attaching the model to an anchor

Conclusion

Part 7: Anchors (WebXR with Babylon.js)

What are Anchors?

Adding anchors to the scene

Interacting with the controller

Adding the Anchor

Cleaning up

Adding observeAnchors to the scene

Conclusion

Part 6: Animating a Mesh (WebXR with Babylon.js)

Essentials & Similarities

Animating a box

These are the steps to accomplish our goal:

Adding the animation to the scene

Conclusion

Part 5: Input/Controllers & Ray Casting (WebXR with Babylon.js)

Ray Casting

What is a Ray Cast?

How does it work?

Hit Testing vs. Ray Casting

Performing a Ray Cast from a Controller Input Source

Controller Detection and Interaction

Adding the Controller Selection to the scene

Conclusion

Part 4: Hit Testing (WebXR with Babylon.js)

What is a Hit Test?

How it works

Use cases

Adding a Mesh as a marker

Performing a Hit Test

Adding Hit Testing to the scene

Conclusion

Part 3: Meshes & Materials (WebXR with Babylon.js)

What is Material?

What is a Mesh?

Babylon.js splits meshes into 4 broader categories.

Creating a Box Mesh

Adding the box to the scene

Conclusion

Part 2: Plane Detection (WebXR with Babylon.js)

What is plane detection?

Prerequisites

Assisted Space Setup

Registering the plane detection feature

Performing the Plane Detection

The complete code

Mesh Visualisation

The complete code

Adding Plane Detection to the scene

Conclusion

Welcome to the Exciting World of WebXR and Babylon.js

What Is WebXR?

Browser support

Why Babylon.js?

Your Learning Journey

Prerequisites

Setting Up Your Environment

Testing Your Setup

The immersive web emulator for the browser

The base template

Defining possible session modes for WebXR.

Defining possible reference space types for WebXR.

Creating our main class and variables

Error Handling for WebGL Support

Setting Up the Canvas and Engine

Creating the Scene

Debugging and XR Configuration

Initializing the XR Experience

Adding `observeAnchors` to the scene