Trying Out Local Image to 3D Model Generation AI Models

The area of AI that has fast acceleration of improvements is content generation. One of my dream projects would be a simple 2D RPG video game, which would require one long period of content creation for the game design phase. Fortunately, the advancement of AI image generation, instructed image editing, and now even converting 2D image to a draft of 3D model in mesh with textures for 3D editors like Unity or Blender. I've tried out beginner lessons of using Blender to start building some object from start and continuously refining shapes and adding textures and lighting. The object editing is obviously the most time-consuming. But nowadays if we have done 2D sketches of object design, we can jump start with an early draft of the 3D model. And software like ComfyUI is an application with
graphical-user-interface using the community-defined workflows built with multiple nodes of functions or AI models, where we can adjust settings and connect to other nodes as sequence of processing.

Thanks to PixelArtistry Youtuber introduction, there are already some tested ComfyUI workflows that help download necessary AI models and use Microsoft's shared Trellis 2 to produce some acceptable 3D model draft. The quality of 3D model produced is pretty good for simple, distinct objects like . Yet complicated objects with many shapes would get pretty rough result, for example, this anime girl in 1024 size. And the alternative is from TripoSplat which really has its existing professional AI services to produce better result than this model with holes using TripoSplat open-source AI model. There are options to increase steps and use different algorithms to process the images, but all those trial and errors would be time-consuming, especially when I only have an average consumer-level GPU of Nvidia GTX 4070 chipset and 12 VRAM.

The other efficient usage of 3D modeling to produce rendered graphics is the production anime videos. Back in the old days, manga, the Japanese comic book, was a set of on computer 2D illustrations organized and printed out in mass production. Before the final illustration, there must be many drafts of illustrations to present effects and movements in different angles and sizes. After the advancement of graphical processing, objects and characters can now be first built in 3D models internally first. For specific scenes, move body parts and adjust expressions and effects if necessary. And move the camera angle and certain distance to better suit the scene's main focus without having to redraw everything. And professional animation software can better program certain objects' animations, and keep many in sync. Overall, 3D modeling has been helpful modern technique to simplify animation videos.