I Tried NeRF This Week and Here's What I Learned.

So this week at my PMW internship I had to research 3D reconstruction methods. I went through COLMAP, NeRF, and Gaussian Splatting.
COLMAP takes multiple photos of a place and builds a 3D point cloud from them. basically it finds matching points across photos and figures out the 3D structure. I think this could work well for PreserveMy.World to reconstruct heritage sites from photo collections.
NeRF is different. it trains a neural network on images and the network learns to show the scene from any new angle. quality is very good but it needs a GPU and takes time. I actually ran a tiny NeRF experiment on Colab this week. built a small neural network that takes 3D points as input and outputs color and density at each point. got a scatter plot as output showing density values across 100 random points. was pretty cool to see it actually work.
Gaussian Splatting is the fastest for rendering. instead of a neural network it places 3D blobs in space and optimizes them to match the photos. renders in real time which I think makes it the best option for PreserveMy.World since users would want to view preserved locations in real time.
overall this week was a good intro to 3D ML. NeRF experiment was simple but it helped me understand how the method actually works at a basic level. next step would be to train it on actual images instead of random points.
PreserveMy.World is working on preserving heritage sites digitally and these methods are directly relevant to that mission.