DEV Community

vmodal_ai
vmodal_ai

Posted on

Building a 3D Gaussian Splatting Viewer with Flutter for Immersive AR/VR

Building a 3D Gaussian Splatting Viewer with Flutter for Immersive AR/VR

3D Gaussian Splatting represents captured scenes using collections of 3D Gaussian primitives. It can produce highly detailed novel views and is increasingly interesting for immersive applications.

Pipeline 

Images / Video
      ↓
3D Reconstruction
      ↓
Gaussian Splat Model
      ↓
GPU Renderer
      ↓
Flutter Viewer
      ↓
AR / VR Experience
Enter fullscreen mode Exit fullscreen mode

What Is a Gaussian Splat?

A Gaussian primitive can contain:

  • position
  • scale
  • rotation
  • opacity
  • color
  • view-dependent appearance information

Large collections of these primitives form a scene.

Flutter Architecture

Flutter should manage:

  • navigation
  • controls
  • loading state
  • settings
  • application state

A native/GPU renderer should handle:

  • camera matrices
  • projection
  • sorting
  • rasterization
  • GPU buffers 
Flutter
  ↓
Viewer Controller
  ↓
Native / GPU Renderer
  ↓
Gaussian Splat Data
  ↓
GPU
Enter fullscreen mode Exit fullscreen mode

Loading a Scene

Use an asynchronous pipeline:

Download
  ↓
Validate
  ↓
Cache
  ↓
Decode
  ↓
GPU Upload
  ↓
Render
Enter fullscreen mode Exit fullscreen mode

Do not block the UI while a large scene is being prepared.

Rendering Pipeline

Conceptually:

3D Gaussian Data
      ↓
Camera Transform
      ↓
Projection
      ↓
Visibility / Depth Processing
      ↓
Gaussian Rasterization
      ↓
Frame
Enter fullscreen mode Exit fullscreen mode

AR Integration

To place a splat scene into AR:

AR Camera Pose
      ↓
Coordinate Conversion
      ↓
World Transform
      ↓
Gaussian Scene
Enter fullscreen mode Exit fullscreen mode

The scene coordinate system must be aligned correctly with the AR coordinate system.

VR Integration

Stereo rendering requires separate views:

             Scene
            /                ↓       ↓
       Left Eye  Right Eye
           ↓       ↓
        Display  Display
Enter fullscreen mode Exit fullscreen mode

Performance

Large scenes can be expensive. Consider:

  • level of detail
  • spatial culling
  • GPU sorting
  • compression
  • streaming
  • quality presets
  • device-specific limits

For example:

Low → Medium → High → Ultra
Enter fullscreen mode Exit fullscreen mode

Memory Management

Large scenes may consume substantial RAM and GPU memory. Explicitly release resources when a scene is no longer needed.

Conclusion

Gaussian Splatting is primarily a rendering challenge. Flutter is well suited to the application shell, while a native GPU renderer can handle intensive splat rendering. This separation provides a practical path to immersive AR/VR viewers.

Useful Links

SDK Flutter: https://github.com/v-modal/vmodal_sdk_flutter

SDK Android: https://github.com/v-modal/vmodal_sdk_android

Discord: https://discord.gg/K72z28KUx

Top comments (0)