Hello! I'm StudioMidori. I'm currently working on a co-op game (tentatively titled "Engawa Summer") using Unreal Engine 5, aiming to recreate the vibe of '90s summer vacation games.
In this series, "Rebuilding Summer," I plan to document the technical aspects of my development process—sharing everything from successes to failures, in a detailed yet sometimes brutally honest way.
For the first installment, the topic is an eternal struggle for solo developers: "How do you get 3D assets?" Let me give you the conclusion upfront. I generated over 100+ assets using only my Mac and local AI, with zero extra costs. I'll be sharing the setup and the pitfalls I ran into.
Setup: A Two-Stage Pipeline from Image Generation to "Image $\rightarrow$ 3D" Conversion
I used entirely free tools that run locally.
| Stage | Tool Used | Notes |
|---|---|---|
| ① Reference Image Generation | ComfyUI + SDXL (RealVisXL) | Generated "product photos," like a "floating 1990s round mailbox on a plain white background." |
| ② Image $\rightarrow$ 3D Conversion | Hunyuan3D-2 (fp16, ~4.9GB) | Generates a 3D mesh (GLB) from a single image. |
| ③ Execution Environment | Apple Silicon Mac / MPS | Did not use any cloud APIs whatsoever. |
While Hunyuan3D feels like it's heavily geared towards CUDA (NVIDIA), it actually runs on Apple Silicon's MPS! This was the part I was most worried about, honestly. I have a history of other 3D models crashing spectacularly due to unimplemented operators on MPS... But this time, it completed successfully. Nice job!
Here are my measured results:
-
Generation Time: Approximately 6–8 minutes per item (
octree_resolution256–320). -
GPU Utilization: Device Utilization 100% during generation (checked with
ioreg. Proof that it wasn't slacking off). - Output Mesh: For a single vending machine, approximately 709,000 vertices / 350k triangles (I'll explain why this is an issue later).
The Quality is Determined by 80% of the "Input Image Creation"
After trying it out, I really felt that the quality of the 3D output is almost entirely determined by the input image. After a lot of trial and error, I settled on the following recipe.
- For boxy objects, shoot from the front view / for cylinders or spheres, shoot at a 45-degree angle (to help it correctly estimate the shape).
- Include the prompt "levitating in an empty white void, nothing underneath". → If you don't include this, the floor and shadow under the object get materialized along with the ground, leading to things like vending machines with wings popping into existence.
- Strictly negative prompt for
podium, plinth, table, shadow(If you leave it up to its own devices, product photos will automatically put items on display stands). -
Do not generate any text (
no text+ negative prompt forchinese characters, garbled text).
The "text" part was the hardest nut to crack. When I tried to get Japanese characters on a sign, the image AI confidently fabricates something that looks like "Chinese characters." So, I gave up on it early and decided to generate all signs as plain white surfaces, and then apply the correct font myself later in the engine. ...I realized later that this decision was "absolutely correct from a copyright perspective," but that's another story for another time.
▼ Input Image (Left) vs. 3-View Silhouette of Generated 3D (Right)
My Epic Failures (This Part is the Most Useful)
It wouldn't be honest for a technical article to only show perfect success stories, so I'm going to share some reproducible failures.
Failure ①: Thin or Small Objects Get "Pancake-fied"
Take the folding phone that symbolized the Heisei era. The result was a thin, flat cracker where the depth information had died. No matter how much I tweaked the prompt, I couldn't get any thickness.
Cause: Single-view image-to-3D models tend to break down when dealing with thin or small objects because they have to guess the unseen sides.
Solution: I decided to accept that for things like phones, pagers, or cassettes—things that are inherently thin—I shouldn't rely on the AI at all. Instead, I figured out it was better to create them later using simple box primitives in the engine. It’s about knowing what tool to use where.
Failure ②: Statues "Multiply" Without Permission
When I asked for just one Jizo statue, two appeared side-by-side, and they somehow merged into some kind of two-bodied thing.
Solution: I explicitly added one single ... , only one figure to the prompt and added multiple, two to the negative prompts. This worked for the guardian dogs and the Inari foxes too; a simple phrase like that brought them back to being single units. If you leave it up to the AI, it tends to be too helpful by adding extra things, so it's key to clearly stating, "Just one is fine."
Failure ③: Code/Antennas "Scatter into Space"
When I generated a game controller, the cable broke apart and floated around like spaghetti in mid-air.
Solution: Exclude thin attachments (like cords, antennas, or strings) from the prompt. Just generate the main body, and add the wires later.
Yield Rate is About 85%. But a Big Pitfall Awaited.
When I limited my focus to solid objects (houses, appliances, statues, cars, buildings), the yield rate was generally around 85%. For "wheeled" items like tricycles or tractors, the wheels seemed to be strong structural hints, resulting in surprisingly clean shapes.
Even organic matter (animals) formed shapes better than I expected.
However, after inspecting the 66 generated items, one fact became clear.
The generated GLBs lack UVs, normals, and materials. They are all just flat gray blobs.
This means I can't do the common sense thing of "applying a single texture." This is the next mountain to climb.
At this point, I thought that I could just apply color collectively on the engine side. To get straight to the conclusion: this assumption will be overturned later. What exactly went wrong, and how, I'll cover in the next post.
Summary and Next Preview
- We can mass-produce game 3D assets for free using just a Mac + local AI (with an estimated yield of 85% for simple objects).
- The quality is 80% dependent on the input image. When prompting, I'll float it in empty space and ask for figures "one by one," avoiding text generation.
- The output is a gray mesh without UVs or materials. The next challenge is figuring out how to colorize this.
Next time, I plan to share everything—including Python scripts for any snags—on importing these 100 mass-produced assets into Unreal Engine and making them actually functional. It will be titled "The episode where we breathe 'color' (not life) into the gray masses."
...Though I really want to say that, I need to give you a heads-up honestly. In this next post, I will completely rebuild my coloring strategy.
I'm updating the progress daily on X (@StudioMidori). If you don't mind, I would be super grateful if you could follow along and watch as summer gets rebuilt one pixel at a time.
――StudioMidori
Top comments (0)