DEV Community: J

Building a Reusable WebGPU & Rust Engine: Part 5 — Implementing Lighting Systems

Fri, 30 Oct 2026 14:34:15 +0000

It’s time to bring depth and atmosphere into our scene through Dynamic Lighting. In a typical rendering pipeline, lighting also works closely with shadows.

The light types I am aiming for are:

Point Light
Directional Light
Spot Light
HDRI Light Analytical lights (point, directional, and spot) use uniform buffer data. In contrast, HDRI lighting is image-based (IBL) and requires a cubemap texture (texture_cube<f32>) and sampler rather than a standard uniform buffer.

Rust Implementation: lighting.rs
In lighting.rs, we create a unified structure to upload the light parameters to the GPU:

light_type (identifying which mode to evaluate)
direction
inner_cone and outer_cone (for spot light boundaries)
color, rangle, intensity (basic things about light)

Just like we did for the depth buffer and post-processing passes, we create a GPU buffer to hold the light information, construct a bind group layout and bind group, and instantiate the light controller in the scene.

WGSL Shader Computation: shader.wgsl
Next, we update the fragment shader to compute illumination based on the active light_type:

Directional Light->> Provides incoming light ray directions (vector L) uniformly everywhere across the scene, independent of position.
Point Light->> Computes the direction from the fragment's world position toward the light position (vector L). It uses an inverse-falloff calculation to dim the light over distance, using clamp to avoid infinite brightness when objects get too close.
Spot Light (Cone Light)->> Calculates the dot product between the light vector vector L and the negative spot direction to get the cosine of the angle. This angle is then compared against the cosine of the inner and outer cone limits to smoothly attenuate the beam.

Diffuse Shading & Material Interaction
To calculate diffuse shading, we compute the dot product between the surface normal vector and the light direction vector, clamping negative values to zero.
Finally, lighting interacts directly with materials. When returning the surface color, we combine all terms together by multiplying:
Final Color = Base Color * Light Color * Light Intensity * Diffuse * Attenuation

Image-Based Lighting (HDRI) and PBR Integration
An HDR image is loaded as an environment texture and mapped using spherical coordinates to project across the scene. However, real-time renderers do not use random sampling at runtime to compute the light. Random sampling is meant for offline path-tracers and causes heavy frame lag and noise.
Instead you prefilter once into:

A diffuse irradiance cubemap that supplies smooth ambient light from all directions.
A pre-filtered specular cubemap that stores reflections blurred across different roughness levels.

HDRI requires a Physically Based Rendering (PBR) workflow to look correct. Without physical material properties like roughness, metallic values, and viewing-angle Fresnel reflection, an HDR image merely acts as a flat ambient tint:

Roughness determines which blurred reflection level is sampled.
Fresnel calculates how reflective the surface becomes at grazing angles.
Metallic controls whether the surface reflects dielectric colors or tinted metallic highlights.

Floating-Point Texture Pipeline Setup
Standard textures clip brightness values at pure white (1.0). Because HDR images store illumination values far higher than 1.0, precision must be maintained throughout the pipeline:

The HDR texture is uploaded using a high-precision 16-bit floating-point format (Rgba16Float).
The offscreen scene texture and the 3D scene pipeline color target format are also set to Rgba16Float instead of the default window surface format. This prevents bright HDR highlights from clipping during the lighting pass.
The post-processing pass reads this 16-bit floating-point scene texture, applies tone mapping to compress the high range down to displayable limits, and outputs the final color to the standard screen format.

Next PBR implementation:

F0 = mix(vec3(0.04), base_color.rgb, metallic)
Fresnel (Schlick)
GGX normal distribution
Smith geometry

Because covering the full pipeline takes significant space, we will wrap up analytical lighting and basic PBR here, and continue with Image-Based Lighting (IBL) environment integration in the next part!

Load the HDRI+PBR texture working together

Building a Reusable WebGPU & Rust Engine: Part 4 — Implementing Depth Buffering and a Post-Processing Pass

Sat, 24 Oct 2026 13:02:05 +0000

part1 link : Windowing & Pipeline Architecture
part2 link : Loading Custom Geometry & WGSL Shaders
part3 link : 3D Camera System & Hot-Reloading Shaders

Upgrading to True 3D
With basic rendering working, the next milestone is rendering a proper 3D cube without visual artifacts like self-occlusion or inverted faces. To achieve this, our engine requires a critical subsystems:

Depth Buffering (Z-Buffer): Ensures closer surfaces correctly obscure farther ones.

On top of depth testing, I am also adding an extra architectural feature:

Post-Processing Pass: Feeds rendered scenes into off-screen textures for screen-space effects and presentation.

Let’s start by breaking down how wgpu manages depth.
Depth testing requires two GPU resources and two pipeline stages working in unison.

1.. The Core GPU Resources

depth_texture: The raw block of VRAM allocated on the GPU via device.create_texture(). Unlike a standard color texture storing RGBA values, each texel stores a normalized distance metric between 0.0 (near plane) and 1.0 (far plane).
depth_view: The access handle created via depth_texture.create_view(). The GPU uses this view as an active attachment to read and write depth values during rasterization.

Lifecycle Note: These resources are typically allocated once at startup, and re-allocated only when the window resizes to match the new swapchain dimensions.

2.. The Two Configuration Stages
(1). RenderPipeline (The Rules)
Tells the GPU how to test and write depth values. Configured once during initialization.

format: Matches your depth texture format (e.g., wgpu::TextureFormat::Depth32Float).
depth_write_enabled: true: Allows closer pixels to overwrite old depth values in VRAM. -depth_compare: wgpu::CompareFunction::Less: Keeps pixels that are closer to the camera and discards anything behind them.

(2). RenderPassDescriptor (The Execution)
Tells the GPU where and when to write depth data. Configured every frame.

depth_stencil_attachment: Binds your depth_view as the active target for the pass.
load: wgpu::LoadOp::Clear(1.0): Clears the buffer to 1.0 (furthest possible distance) at the start of each frame so new geometry passes the test.
store: wgpu::StoreOp::Store: Saves the written depth values during the pass so subsequent draw calls can test against them.

Implementing the Post-Processing Pass
Post-processing requires setting up an off-screen render target and the pipeline resources to sample it.

1. The Core Setup
The post-process pass needs a bind group layout (the interface) and a bind group (the resources). In State, post_bind_group_layout declares binding 0 as a filterable texture_2d and binding 1 as a filtering sampler, both visible to FRAGMENT — matching post_process.wgsl exactly.
post_bind_group binds the off-screen scene_view and scene_sampler to that contract.

post_pipeline is built by the same create_render_pipeline helper as the main pass, passing depth_format: None (no depth testing in pass 2) and use_vertex_buffer: false, because the vertex shader generates a full-screen triangle from@builtin(vertex_index). post_pipeline_layoutwraps the bind group layout above.

2. The Hot-Reload Trade-Off & Safety Boundaries
Storing pipeline handles on State allows shader code to recompile dynamically at runtime:

Fallback & Standalone Safety (include_str!): The initial pipelines are compiled using include_str!, embedding the default WGSL sources directly into the final binary. The engine can always run standalone without external file dependencies.
Disk-Based Watcher: Runtime reloads use ShaderWatcher (powered by the notify crate) to detect changes and re-read files from disk relative to the current working directory.
Non-Crashing Error Handling: When a file changes,reload_shaders builds candidate pipelines inside a wgpu::ErrorFilter::Validation scope via push_error_scope. If compilation fails due to a typo or syntax error, the engine logs the compiler output and retains the current active pipelines instead of crashing.
Interface Immutability: Hot-reloading only swaps out the shader modules and internal pipeline logic—it does not rebuild bind group layouts. As long as @group and @binding slots match the original contract, you can iterate on math, colors, and post-processing algorithms completely live.

Proper 3D Depth Testing with a Vignette Post-Processing Pass

Thanks for reading! Check out my industrial Rust projects available on the Microsoft Store:
Download on the Microsoft Store

Building a Reusable WebGPU & Rust Engine: Part 3 — Implementing a 3D Camera System & Hot-Reloading Shaders

Sat, 17 Oct 2026 13:48:04 +0000

It’s been a while since my last update! I received some great feedback and architectural suggestions, especially around asset loading and pipeline optimization. I plan to integrate those ideas step by step.

Before jumping into advanced features like shader hot-reloading or multi-threaded asset streaming, we need to solve a fundamental problem: moving from flat 2D rendering to a full 3D viewport. Today, we are focusing on setting up a Pinhole 3D Camera System.

1. Why Do We Need a Camera? (Normalized Device Coordinates)

If you render a mesh in wgpu without a Camera View or Projection matrix uniform in your WGSL shader, your vertex positions are passed directly into WebGPU's Normalized Device Coordinates (NDC) clip space.

In wgpu, NDC is a bounded clip space where:
X and Y range from [-1.0, 1.0] Z ranges from [0.0, 1.0]

Without a perspective matrix to perform perspective division (scaling geometry based on its distance along the Z-axis), your scene renders as a flat, orthographic box. Vertices outside this [-1.0, 1.0] range get clipped instantly, and objects don't get smaller as they move further away.

2. Building a Standard Pinhole Camera

To achieve realistic 3D depth, we implement a classic pinhole camera model using two core matrix transformations:

Projection Matrix (cgmath::perspective): Constructs a Field-of-View (FOV) frustum that scales coordinates based on distance and applies aspect ratio correction so objects don't stretch when resizing the window.
View Matrix (cgmath::Matrix4::look_at_rh): Transforms world space relative to the camera's eye position, target orientation, and "up" vector using right-handed coordinate space (_rh).

note: cgmath was designed for OpenGL (-1.0 to 1.0 Z-range). I multiply by OPENGL_TO_WGPU_MATRIX to map Z to (0.0 to 1.0).
let camera_matrix = OPENGL_TO_WGPU_MATRIX * proj * view;

3. Passing Camera Uniforms to WGSL

To get our camera matrix into the shader, we set up an explicit layout contract using wgpu bind groups:

camera_controller.bind_group_layout : Tells WebGPU the exact shape and memory visibility of the uniform data before it enters the shader (@group(0) @binding(0) var<uniform> camera: CameraUniform;).
render_pipeline_layout: Registers the camera layout into slot 0 so the pipeline expects camera data in @group(0) during draw calls.

Modular Binding Strategy
Structuring your bind groups explicitly keeps your engine scalable. When you add materials, textures, or lights later, you simply bind them to higher group slots without breaking your camera setup:

// Group 0: Camera Uniforms (Updated once per frame or camera move)
@group(0) @binding(0) var<uniform> camera: CameraUniform;

// Group 1: Material & Textures (Updated per mesh/material)
@group(1) @binding(0) var t_diffuse: texture_2d<f32>;
@group(1) @binding(1) var s_diffuse: sampler;

// Group 2: Lighting Data (Updated when scene lights change)
@group(2) @binding(0) var<uniform> light: LightUniform;

What’s Next?
With the camera matrix updating uniform buffers every frame, we can now freely orbit, pan, and zoom around any loaded 3D mesh. I will leave the details of those features for later.

Here we get a slightly interesting image, cube with perspective change but no depth.

Bonus: Shader Hot-Reloading

Restarting the application every time you tweak a color or lighting equation in WGSL slows down iteration fast.

Instead of pre-compiling shaders, we can use notify to watch the -/any_name/ folder and rebuild the RenderPipeline at runtime whenever a .wgsl file is saved.

How It Works Under the Hood:

OS-Level Asynchronous Tracking: I use notify::RecommendedWatcher to hook into the operating system's native file-notification API. This monitors file changes in the background without blocking the rendering thread.
Main-Thread Channel Receiver (Receiver<Result<Event>>): The background watcher sends file-system updates through a thread-safe mpsc channel. The receiver side stays available on the main thread.
Triggering the Reload on Redraw: During the RedrawRequested step inside our main event loop, we non-blockingly check the channel using watcher.rx.try_recv(). If a file-modification event is detected, state.reload_shaders() reads the updated .wgsl file from disk and replaces the active RenderPipeline on the fly.

and we can change the shader without close the application!

Thanks for reading! If you're working on similar systems, check out my personal projects:
Building industrial 3D vision or AI pipelines? |
Check out

— designed to help benchmark, test, and validate your detection networks against challenging optical geometry.

Building a Reusable WebGPU & Rust Engine: Part 2 — Loading Custom Geometry & WGSL Shaders

Fri, 09 Oct 2026 05:09:30 +0000

In Part 1, we talked about the CPU event loop and established our foundational wgpu state. Now, we expand our engine to load 3D geometry (.obj files) from disk and feed it through a custom WGSL Shader Pipeline.
old-post link

Writing WGSL Shaders

Shaders are the small, highly parallel programs that run directly on the GPU to render 3D geometry and color pixels.

In wgpu, we primarily work with two types of shaders:

Vertex Shader (@vertex): Processes incoming vertex positions and attributes. Its primary job is converting 3D coordinates into Clip Space coordinates.(homogenous coordinates used by the GPU rasterizer)
Fragment Shader (@fragment): Runs once for every pixel (or fragment) covered by a primitive (like a triangle) to determine its output color.
We will go back to this part in short furture! it’s very useful process.

(Note: traditional APIs had geometry shaders, WebGPU omits them to ensure consistent cross-platform performance.)

When loading external 3D models (like an .obj file), here I pass three core vertex attributes to the vertex shader:

Position: The 3D spatial coordinates [x, y, z].
Texture Coordinates (UV): Mappings used to align 2D textures onto 3D surfaces.
Normals: Surface direction vectors, essential for calculating real-time lighting in the fragment shader.

Preparing Memory Layouts & CPU-to-GPU Buffers

To pass .obj model data from the CPU to GPU memory, we establish two key data structures:

1. Vertex custom data struct (wgpu::VertexBufferLayout & wgpu::VertexAttribute):

position — [f32; 3]
tex_coords — [f32; 2]
normal — [f32; 3]

Defines how memory is ordered inside the vertex buffer. Using wgpu::VertexAttribute, we specify memory offsets for each field (Position, UV, Normal) so the GPU knows exactly where each attribute begins in raw binary data.

2. Mesh custom data struct (CPU Side):

Once the .obj file is loaded into your Vertex structs, you upload them to the GPU. You create a Mesh struct to hold onto those GPU handles:

Vertex Buffer: Stores the array of packed vertex data.
Index Buffer: Stores the vertex indices, allowing triangles to reuse shared vertices to save GPU memory.
Index Count (indices.len()): Tracks the total element count required by render_pass.draw_indexed().

3. Connecting Everything in the Render Pipeline

[ .obj File & .wgsl Shader ]
              │
              ▼
    Load from Local Disk
              │
              ▼
  device.create_shader_module()  ──► Compile WGSL Code
  Mesh::load_obj()               ──► Upload Vertex/Index Buffers to GPU
              │
              ▼
 device.create_render_pipeline() ──► Reference Vertex::desc() for Memory Layout
              │
              ▼
 [ Store Mesh & Pipeline in State ] ──► Ready to Draw in state.render()

Load Assets: Read the .obj file into a Mesh struct and compile the WGSL shader code using device.create_shader_module().
Build the RenderPipeline: Create the pipeline via device.create_render_pipeline(), setting the @vertex and @fragment entry points. Pass description Vertex::desc() into the pipeline setup so wgpu render_pipeline knows how to map the vertex buffer straight into shader inputs.
Execute during Render: Store both the RenderPipeline and Mesh inside your State. Inside render() loop, bind the pipeline and draw the mesh buffers

Just like in Part 1, in the end the exciting result: we can load a 3D box into the scene using our custom shader—even without setting up a camera yet!

Thanks for reading, here is my Industrial Rust Projects:
Building industrial 3D vision or AI pipelines? Check out ForgeOptics on Gumroad for high-performance Rust ray-tracing, physical sensor noise, and synthetic dataset generation.

Building a Reusable WebGPU & Rust Engine: Part 1 — Windowing & Pipeline Architecture

Tue, 06 Oct 2026 12:52:27 +0000

Building a modular render engine or simulator template requires a clean separation between CPU-bound application logic and GPU-driven rendering pipelines.

In this post, I will track the process of building a lightweight, reusable render template in Rust using wgpu and winit.

Nowadays, we can utilize AI to handle implementation details, but as creators, we still need to review and master the architecture from a high level. Understanding the top-level structure helps us learn faster, validate the AI's output, and ensure our systems remain robust and maintainable.

1. Project Setup & Dependencies
Initialize a new Rust project and add wgpu (for graphics API abstraction) and winit (for cross-platform window management):

additionally: pollster: A minimal async executor used to block on GPU async calls during initialization.

2, Host Application Setup (CPU Side)
Before setting up wgpu, we build the host CPU process. This handles input, windowing, network requests, and overall application state separately from the renderer.

The event loop is the key structure on the CPU side. It drives the entire application lifecycle and coordinates user interactions, window events, and graphics updates.

       +------------------------------------+
       |         winit::EventLoop           |
       +------------------------------------+
                         |
                         v
  +----------------------------------------------+
  |  Event Handling (Main Loop)                  |
  |                                              |
  |  * WindowEvent::CloseRequested -> Exit App   |
  |  * WindowEvent::Resized        -> Resize GPU |
  |  * Other Events                -> Ignore     |
  +----------------------------------------------+

By isolating CPU application logic from rendering calls, you keep the host process clean and modular before bringing in wgpu.

3. GPU Pipeline Architecture (wgpu)
Once the window exists, we initialize the wgpu state. Rather than focusing on syntax, think of the GPU initialization as connecting five key structural primitives:

+-----------------------------------------------------------------------+
|                              wgpu State                               |
|                                                                       |
|  [ Surface ]  <--->  Bridge between the OS window and the GPU         |
|  [ Device ]   <--->  Virtual GPU interface (creates pipelines/buffers)|
|  [ Queue ]    <--->  Command channel to send work to physical hardware|
|  [ Config ]   <--->  Surface settings (pixel format, dimensions, vsync)|
|  [ Size ]     <--->  Tracks viewport bounds for dynamic resizing      |
+-----------------------------------------------------------------------+

As your engine grows, this structure expands to hold high-level rendering primitives like , , and .

The Core Execution Loop
Once initialized, the GPU pipeline operates around two primary execution routines:

[resize]

Fired whenever the window dimensions change. It updates the and reconfigures the so the rendering context stays locked to the window bounds without distortion.

[render]

The per-frame execution sequence:
Acquire Frame: Request the current frame buffer (output) and construct its texture view (view).
Encode Commands: Open a command encoder and launch a scoped Render Pass.
Manage Ownership: Crucially, the render pass must finish and drop its resource locks within its localized block {} before submission.
Submit & Present: Pass the encoded commands to the for execution on hardware, then display (present) the output frame.

After that, the "event handler" will connect "AboutToWait" and "RedrawRequested", invokes the [render] sequence

Finally showing the exciting result (a colored canvas- yeah!!!):

Bridging the Sim-to-Real Gap in 3D Structured Light Vision with Physics-Based Ray Tracing

Sun, 27 Sep 2026 13:57:11 +0000

The Problem with Geometric Z-Buffers

Standard depth-map generators output perfect geometric distances. However, real-world 3D vision relies on phase-shifting optical projections.

To solve this, ForgeOptics — a high-performance 3D structured light simulator specifically for computer vision researchers and robotics engineers.
Instead of generating ideal depth maps, ForgeOptics simulates the entire physical phase-shifting optical process:

Projecting sinusoidal fringe patterns
Path-tracing PBR surface reflections and scattering
Decoding phase shifts into ground-truth 32-bit floating-point depth maps Check out the full workflow below—covering GUI parameter tuning, Sigmoid sensor noise injection, and dataset generation:

Key Capabilities at a Glance |
1) High-Performance Ray-Tracing Engine, Built natively in Rust and powered by LuisaRender for fast, denoised, near-real-time previews and rapid dataset generation.

2) Dual Interactive & Automated Workflows

Interactive GUI Mode: Visually import 3D meshes (.obj, .glb, .gltf, .stl), edit PBR materials on the fly, adjust projector frequency, and configure instance segmentation IDs.
Headless CLI Mode: Every GUI parameter—projector, camera, lighting, and materials—is fully exposed in JSON format for automated background execution and batch dataset generation.

3) Real-World Degradation & Point Cloud Verification
Introduce custom Sigmoid sensor noise in camera settings to simulate physical hardware degradation.
Directly export reconstructed 3D PLY point cloud data, 32-bit floating-point depth maps, and exact camera intrinsic matrices for immediate downstream processing.

Try the Free Sample Dataset

Before buying or deploying, you can download and inspect free sample datasets containing full 32-bit depth maps, PLY point clouds, and camera intrinsics:
💾 Free Sample Dataset on GitHub: ForgeOptics GitHub Repository

🤗 Hugging Face Dataset Hub: ForgeOptics Structured Light Samples

Commercial Licensing & Downloads

Ready to automate synthetic dataset generation for your AI training pipeline or 3D vision algorithms?

🛒 Purchase Commercial Binary Licenses: ForgeOptics on Gumroad

Why Vision Models Fail on Glass: A Synthetic Benchmark for Transparent Objects

Fri, 25 Sep 2026 15:27:53 +0000

*Transparent objects like glass, acrylic, and water are the ultimate blind spots for modern Computer Vision models. *

Whether you are training Depth Estimation, 3D Reconstruction (NeRFs/3DGS), Segment Anything (SAM), or Object Detection models, transparent surfaces ruin feature matching and depth sensors. RGB light passes right through, reflections distort geometries, and standard datasets completely lack labeled ground truth for transparency.

To solve this, we built a synthetic benchmark dataset tailored specifically for training and evaluating vision models on transparent materials.

🛑 Why Traditional CV Pipelines Break on Glass

Depth Sensors Fail: ToF (Time-of-Flight) and active stereo depth sensors pass through or scatter when hitting optical-grade glass.
Refraction & Reflection: Standard RGB feature extractors detect background features behind the glass or virtual highlights on the surface—leading to incorrect bounding boxes and false segmentations.
Lack of Ground Truth Data: Capturing real-world paired RGB-D images of transparent objects with pixel-accurate normal maps and segmentation masks is near impossible manually.

🎬 Synthetic Data Generation Benchmark (Demo)

Watch our synthetic pipeline render optical-grade reflections, precise refraction passes, and exact ground-truth depth maps in real-time:

📊 What's Inside the Benchmark Package

To help researchers and engineers train robust models, we synthesized a specialized 3D dataset focused entirely on glass physics and transparent geometries:

Clean Ground-Truth Annotations: Instant access to aligned RGB images, 2D Bounding Boxes (BBox), and precise Instance Segmentation masks.
Physics-Based Refraction: Varied Index of Refraction (IOR) materials—from glass bottles and windows to lab glassware.
Complex Lighting Environments: High dynamic range (HDR) lighting variations to teach models edge detection under extreme specular reflections.
Ready for Model Training: Pre-formatted annotations for YOLO, SAM, and Depth Anything models.

📦 Get the Synthetic Glass Dataset Package

Ready to benchmark your depth estimation models or fine-tune your vision architecture for hard transparent objects?

👉 Download the Glass Benchmark Package on : package1
package2

Discussion

How are you currently handling transparent object segmentation in your vision pipeline? Are you relying on multi-modal sensors, or fine-tuning vision foundation models? Let's discuss in the comments below!

Sythetic data generation

Thu, 24 Sep 2026 15:55:40 +0000

Synthetic data often fails in production because domain gaps

— especially with transparent objects!
I am developing a solution focused on High-Fidelity Validation Assets. By moving beyond "pretty" images to industrial-grade evaluation, we can stress-test models against complex refractions and dynamic lighting while maintaining 100% geometric truth.

Github-link: https://github.com/ForgeCore-Labs/Synthetic-Glass-Transparent-Packaging-Indoor-Pack

FC-babylon-pathtracer — an open-source progressive path tracer built for Babylon.js!

Thu, 24 Sep 2026 04:13:48 +0000

Excited to share FC-babylon-pathtracer — an open-source progressive path tracer built for Babylon.js!
The core focus here is interactive viewport quality: it includes a zero-readback GPU a-trous denoiser that reads first-hit AOVs (Albedo, Normals, Depth) to clean up noise instantly without killing performance.
Highlights: • GPU A-Trous Denoising (edge-aware filtering) • Web Worker BVH construction (SAH Quality) • Live material parameter updates without shader re-compilation • Support for PBRMaterials, StandardMaterials, and environment maps
Quick Start: You can pull it straight into your local Babylon.js setup before the official npm publish:
npm install @babylonjs/core
npm install /path/to/babylon-pathtracer

Git:https://lnkd.in/ghxuyue6