I wanted to touch Gaussian Splatting (3DGS), but every time I tried to start with the paper I got stuck at the math. So I tried running it on Colab first. Once "what goes in, what comes out" was visible on my own screen, the paper was much easier to come back to. This is the record of doing it in that order.
The deeper theory is already covered better by other writers. Here I am only writing about "what runs, in the first 30 minutes."
Quick answer
Yes, a free Google Colab GPU runtime (T4) is enough to train a Gaussian Splatting scene end to end. In this walkthrough, ~5 minutes to download the dataset, 15-30 minutes to train 7,000 iterations, and ~5 minutes to render test views — no prior 3D graphics or NeRF background needed. The walkthrough itself trains on the NeRF Synthetic dataset (which already ships with per-image camera pose data), not on photos you shoot yourself. For a real-world scene, the pipeline needs 30-300 photos plus camera pose data that you generate by running COLMAP first — that step is not covered here. The output is an exportable .ply Gaussian point cloud you can rotate in a browser with SuperSplat.
The setup
| Item | Value |
|---|---|
| Environment | Google Colab (GPU runtime: T4 worked) |
| Training data | NeRF Synthetic dataset (the one the official repo recommends) |
| Time | ~5 min data download + 15-30 min training + ~5 min rendering |
You do not need prior knowledge of 3D graphics or NeRF. If you can read pip install, you can follow along. I'll keep the gotchas at the end.
What is Gaussian Splatting actually doing
Roughly: "Given a set of photos, place millions of transparent ellipsoids (Gaussians) in space such that, from any viewing angle, the rendered view matches the photo of that angle."
Unlike textbook 3D models (collections of triangle polygons), Gaussian Splatting represents the scene as a swarm of points, each with position, color, opacity, and shape. Things that are hard to express with polygons — fur, smoke — come out reasonably natural.
Compared with NeRF (Neural Radiance Fields), the difference is sharper:
| Item | NeRF | Gaussian Splatting |
|---|---|---|
| Scene representation | Neural network weights (black box) | Explicit 3D Gaussian point cloud (exportable) |
| Render speed | Slow (seconds per frame) | Fast (real-time) |
| Train speed | Slow (hours) | Fast (tens of minutes) |
| Compatibility with downstream tools | Poor | Easy to drop into Blender etc. |
If NeRF is "an AI that memorized the scene," Gaussian Splatting is "a point cloud file of the scene." The fact that the output comes out as a file you can hand to other tools later was, personally, the bigger differentiator than the speed.
The data flow, before touching code
Before any code, mapping "what goes in / what comes out":
[Input]
├── Image files (PNG/JPG) 30-300 per scene
├── Camera info (transforms_train.json) Where each image was shot from
└── Initial point cloud (points3D.ply) Present in COLMAP datasets; absent
in NeRF Synthetic — random init works
↓ Training (iterative optimization)
[Intermediate]
Gaussian point cloud: hundreds of thousands to millions
Per point: position (x,y,z), color (RGB), opacity, shape (covariance)
↓ Render
[Output]
2D images from arbitrary viewpoints (real-time)
The thing to notice early: training does not start without the camera-info JSON. "Just a folder of photos" is not enough — you need per-image camera pose data.
If you want to start from real-world photos, you'd run COLMAP (a Structure-from-Motion tool) first to produce that JSON. Today I'm using the NeRF Synthetic dataset, which ships with the JSON already attached. For a first run, that's the easier path.
Running it on Colab: the copy-paste checklist
The official repo (graphdeco-inria/gaussian-splatting) README is thorough but written for serious users. Getting it to work in Colab in one pass needs a bit of decoding. When I tried it straight, the first data-download URL on the README was already broken, and the Google Drive mirror was rate-limited. Below are the cells that finally got me through, in the order I ran them. Open a fresh Colab notebook, set the runtime to GPU (T4), and run from the top.
1. Confirm the GPU is actually attached
If no GPU name shows up, fix the runtime type and reconnect before going further. Every later step assumes a GPU.
!nvidia-smi --query-gpu=name,memory.total --format=csv
import torch
print(torch.__version__, torch.cuda.is_available(), torch.version.cuda)
2. Clone the repo (recursively)
diff-gaussian-rasterization and simple-knn are submodules. Forget --recursive and you'll hit "directory is empty" errors later in the build.
!git clone --recursive https://github.com/graphdeco-inria/gaussian-splatting
%cd /content/gaussian-splatting
3. Build the CUDA extensions
This is the most failure-prone step. Print both the Colab-side CUDA and the PyTorch-side CUDA before installing — if they don't match, the build will silently produce broken binaries.
!nvcc --version | tail -1
import torch; print('torch CUDA:', torch.version.cuda)
!pip install -q ./submodules/diff-gaussian-rasterization
!pip install -q ./submodules/simple-knn
!pip install -q plyfile tqdm
4. Get the dataset
The official tutorial's cseweb.ucsd.edu nerf_synthetic.zip URL redirected to 404 when I tried it. The Google Drive mirror gets blocked by gdown access limits a lot. What worked was pulling just the lego scene from a HuggingFace mirror that has the dataset extracted.
!pip install -q huggingface_hub
from huggingface_hub import snapshot_download
path = snapshot_download(
repo_id='pablovela5620/nerf-synthetic-mirror',
repo_type='dataset',
allow_patterns=['lego/**'],
local_dir='/content/data',
)
!ls /content/data/lego
5. Train
On my T4 (15 GB) run, 7,000 iterations trained without hitting memory limits. NeRF Synthetic has a white background so I pass --white_background.
!python train.py -s /content/data/lego --iterations 7000 --white_background --eval
6. Render
Outputs land in output/<random runid>/. To avoid hunting for the runid manually, grab the latest run programmatically and render from it.
import glob, os
RUN = sorted(glob.glob('/content/gaussian-splatting/output/*'), key=os.path.getmtime)[-1]
!python render.py -m {RUN}
Rendered images appear in output/<runid>/test/ours_7000/renders/. If the lego excavator's shape reconstructs across all the test viewpoints, training worked.
Gotchas (things that genuinely stopped me)
CUDA build fails
If diff-gaussian-rasterization won't build, it's almost always that the Colab CUDA and the PyTorch CUDA are mismatched. !nvcc --version vs torch.version.cuda — print both and you'll see the gap. Reinstall PyTorch against the right CUDA and the build goes through.
Confusion about the initial point cloud (points3D.ply)
I read the COLMAP docs and convinced myself that "every scene needs sparse/0/points3D.ply." Actually that's only for real-world photos run through COLMAP. NeRF Synthetic only needs transforms_*.json and train.py starts from a random initial point cloud. Once I separated "COLMAP-derived datasets need points3D" from "synthetic datasets don't," the confusion cleared.
CUDA out of memory
As Gaussian count grows, VRAM evaporates. In my case, 7,000 iterations on a free-tier T4 (15 GB) ran without issue; pushing toward 30,000 is where I'd expect to hit the ceiling, based on how VRAM use trended as the count grew (I didn't push that far myself). Start at 7,000, confirm it runs, then push iterations up and watch nvidia-smi.
Data URLs are unreliable
Saying this twice because it tripped me up most: the official wget URL redirected to 404, the Google Drive mirror was rate-limited. HuggingFace mirror worked for me, but distribution endpoints come and go. Before anything else, just confirm the dataset actually downloads — that's the most time-saving check.
Viewing the output
After training, the final Gaussian point cloud is at output/<runid>/point_cloud/iteration_<N>/point_cloud.ply. Open it with one of:
-
SuperSplat: browser-based
.plyviewer. Drag and drop, lowest barrier to entry. - Blender + Gaussian Splatting add-on: useful when composing with other 3D scenes downstream.
- The SIBR viewer bundled with the official repo: requires local Linux/Windows.
For a first viewing, SuperSplat is the easiest. Throw the .ply at the browser, rotate it.
After running it once
Before running, I had Gaussian Splatting in my head as "some heavy 3D generation thing." After running it once on Colab, the actual flow — "feed in photos and camera info → iterative optimization runs → point cloud file comes out" — turned out to be not that complicated. Data prep is more work than the model is.
When the 7,000-iteration render came back with the same lego excavator reconstructing from every test angle, that was the moment "ah, this is actually a 3D scene" clicked.
Trying to read the paper first and stalling on the math, vs. running it first and then reading — for me, running first made the paper readable. The math equations stopped being opaque; they were now "ah, this parameter, this calculation." Some people find it uncomfortable to skip the theory first. I'd say: this order is fine.
Where to push next
After the first run goes through, three directions look interesting:
-
Run on your own data — your phone shoots a video, COLMAP produces
transforms.json, train on that. Adds the COLMAP step. - Move off Colab — Colab runtimes die at 12 hours. For serious training, Modal charges by the second and lets long runs survive.
- Read for the parameters — read scomup's Japanese explainer while you adjust learning rates and Gaussian split thresholds. Reading while running settles things in.
FAQ
Does a free Colab GPU actually work for this?
Yes. The T4 GPU on Colab's free tier trained 7,000 iterations without hitting memory limits in this walkthrough. Pushing toward 30,000 iterations is where I'd expect to hit the VRAM ceiling on a 15 GB T4, based on how usage trended as the Gaussian count grew — I didn't personally push that far.
How many photos do I need?
For a real-world scene run through COLMAP, the pipeline typically needs 30-300 photos plus per-image camera pose data that COLMAP generates from them. This walkthrough sidesteps that step by training on the NeRF Synthetic dataset, which already ships with the camera pose JSON attached, so no photos need to be shot for a first run.
What GPU/CUDA setup do I need to watch for?
Set the Colab runtime to GPU (T4) before starting. The most failure-prone step is building the CUDA extensions (diff-gaussian-rasterization, simple-knn): if the Colab-side CUDA version (nvcc --version) and the PyTorch-side CUDA version (torch.version.cuda) don't match, the build can silently produce broken binaries.
How long does the whole run take?
Roughly 30-40 minutes total: ~5 minutes to download the dataset, 15-30 minutes to train 7,000 iterations, and ~5 minutes to render the test views.
Do I need 3D graphics or NeRF background knowledge first?
No. If you can read pip install, you can follow the steps. The theory becomes easier to absorb after you've seen the input/output on your own screen, not before.
How do I view the trained scene afterward?
The final Gaussian point cloud lands at output/<runid>/point_cloud/iteration_<N>/point_cloud.ply. SuperSplat is the lowest-friction option — a browser-based viewer where you drag and drop the .ply file. Blender with a Gaussian Splatting add-on and the SIBR viewer bundled with the official repo (local Linux/Windows only) are the other two options mentioned in the official tooling.
References
- graphdeco-inria/gaussian-splatting (official)
- Gaussian Splatting on Colab (classmethod, JP)
- Gaussian Splatting explainer (scomup, JP)
- SuperSplat (.ply viewer)
- NeRF Synthetic mirror (HuggingFace)
- Modal Labs docs
Top comments (0)