Make a realistic 3D model just by taking photos
Using Apple's tool, you can easily create 3D models.
A turtle, and the captured turtle.
I want realistic 3D models
If you have plenty of usable 3D models, you can use them in AR and game apps. But (in my experience) freely usable 3D models for AR/games are surprisingly scarce — on download sites, the nice content is often paid.
Making 3D models seems hard
And making them yourself — modeling and converting in 3D software — seems difficult.
With Apple's RealityKit, just taking photos makes a model
In 2021 Apple released a tool that builds a 3D model just from photos. It's fairly easy and produces realistic objects. It recognizes the salient object and cleanly separates it from the background and floor.
Method
1. Take photos
With a handheld camera (iPhone is fine), photograph the thing you want in 3D from every direction.
I simply put a stuffed animal on the carpet and shot 360° from the side and from above, covering it like a hemisphere. The conditions:
- in as bright a place as possible
- with the whole subject as large in frame as possible
- shooting frequently so consecutive photos overlap by 70%+
I took 200 photos and gathered them into a folder on my Mac.
2. Download Apple's tool
You can get it from the developer site:
https://developer.apple.com/documentation/realitykit/creating_a_photogrammetry_command-line_app/
Build it in Xcode and a HelloPhotogrammetry file appears in the product folder; open it in Finder to find its location.
3. Run
<path to HelloPhotogrammetry> <input image folder> <output file path, with .usdz>
* Absolute paths from root are required.
This generates a USDZ model. With 200 photos it took about 20 minutes. Click the link below on an iPhone to try the turtle USDZ model in AR:
Toward richer 3D content
Some people upload models made this way to 3D-model sharing sites. It'd be great if 3D models and content grew richer as lots of people scan lots of things.
Originally published in Japanese on Qiita. I build apps with Core ML and ARKit and write about ML/AR. GitHub / X
Top comments (0)