Trying Out Local Image to 3D Model Generation AI Models

Brian Gan — Tue, 01 Sep 2026 21:33:36 +0000

The area of AI that has fast acceleration of improvements is content generation. One of my dream projects would be a simple 2D RPG video game, which would require one long period of content creation for the game design phase. Fortunately, the advancement of AI image generation, instructed image editing, and now even converting 2D image to a draft of 3D model in mesh with textures for 3D editors like Unity or Blender. I've tried out beginner lessons of using Blender to start building some object from start and continuously refining shapes and adding textures and lighting. The object editing is obviously the most time-consuming. But nowadays if we have done 2D sketches of object design, we can jump start with an early draft of the 3D model. And software like ComfyUI is an application with
graphical-user-interface using the community-defined workflows built with multiple nodes of functions or AI models, where we can adjust settings and connect to other nodes as sequence of processing.

Thanks to PixelArtistry Youtuber introduction, there are already some tested ComfyUI workflows that help download necessary AI models and use Microsoft's shared Trellis 2 to produce some acceptable 3D model draft. The quality of 3D model produced is pretty good for simple, distinct objects like . Yet complicated objects with many shapes would get pretty rough result, for example, this anime girl in 1024 size. And the alternative is from TripoSplat which really has its existing professional AI services to produce better result than this model with holes using TripoSplat open-source AI model. There are options to increase steps and use different algorithms to process the images, but all those trial and errors would be time-consuming, especially when I only have an average consumer-level GPU of Nvidia GTX 4070 chipset and 12 VRAM.

The other efficient usage of 3D modeling to produce rendered graphics is the production anime videos. Back in the old days, manga, the Japanese comic book, was a set of on computer 2D illustrations organized and printed out in mass production. Before the final illustration, there must be many drafts of illustrations to present effects and movements in different angles and sizes. After the advancement of graphical processing, objects and characters can now be first built in 3D models internally first. For specific scenes, move body parts and adjust expressions and effects if necessary. And move the camera angle and certain distance to better suit the scene's main focus without having to redraw everything. And professional animation software can better program certain objects' animations, and keep many in sync. Overall, 3D modeling has been helpful modern technique to simplify animation videos.

In comparison to advanced AI agents like Claud Opus/Fable or GLM that are the whole package thinking and directing the use of other knowledge and tools in the background, a tool like ComfyUI is very specific to certain media production tasks and we would be the manager of the tools, settings and workflows. The frequent problems encountered include missing or access to the AI models, bugs in codes of those models, and the hardware compatibility and capability issues. Unless we are the specialists and pros in this department, we are even fortunate to have ComfyUI and community managed models and workflows used within the same software.

Eventually these will improve along with the advancement of AI industry.

Let AI Write the Mac OS App Without Learning Swift

Brian Gan — Tue, 25 Aug 2026 23:26:56 +0000

AI keeps accelerating through the tech industry. Just like many other laid-off tech workers, I feel little miserable how far has AI really affected companies to minimize or eliminate junior to mid level software engineers. Starting last year I've been using the basics of premium GitHub plan with various versions of Claude Code, Gemini, and GPT, to generate bare bones of applications and components, use auto-inspect-and-suggest coding segments, and ask technical questions about certain issues in general applications. Recently I'm trying even more complicated features like generating new project based on description and prompts, and solving problems by exploring the context of entire project.

Latest trial is to find an idea of software project, and prompts AI agents to generate as much as possible. The open-source Windows program Rufus is useful free tool to write some ISO image to a disk drive, for example, bootable Windows installation disc. Check Rufus' screenshot. Cannot find similar free tool in Mac OS. Thus, this application is trying to simulate the basic functions of Rufus. It's unusual for me to choose a desktop app, but I feel such disk utility is missing or unknown compared to vast library of utility programs in Windows during this multiple OS installation disks making. Personally, I get blurry with side projects because not many inspirations and ideas of what to make. This necessary utility would give me push to develop something.

The framework used is Swift UI within MacRufusSwift subfolder. Since I have only little knowledge of Swift and desktop app programming, I started using Claude Sonnet 4.6 initially to generate and fix up the basic structure of the app, and amazingly those prompts alone ate up one month's premium request capacity. So for further modifications, testing and debugging would need assistance.
Since my paid CoPilot premium usage for August was already eaten by early prompts to summarize Rufus written in C++ and generate similar one in Swift for Mac OS, I have to reconsidering paying these AI services for big operations. So the other option is to try running open-source AI models locally. I did installation of Ollama and selected models on my own Macbook Pro: Ollama, its own llama3.1:8b, qwen2.5-coder-7b, gemma4:2b, gemma4:4b, and nomic-embed-text. I'm tied up with these smaller local models because to be developing and running local AI pretty much use up most of my MBP's memory. I'm preparing my PC with mid-level video card to run AI better, so beginning of setup has been painful: replace privacy-stealing Windows 11 with fresh installation of Windows 10, a more trust-worthy OS that still works with most software. Linux is installed on the other drive, but the latter is slower SSD and setup on Linux would take more time for another project. Overall, it's cumbersome to use a large PC, especially now electricity rates and hardware prices are continuously being raised.

llama3.1:8b - More like general technical knowledge assitant. Only briefly used it to resolve coding problems, so not sure how well it performs.
gemma4:2b - Actually pretty good in analyzing the context of codes, and suggesting solutions. After typing the class or function synopsis and functionalities, its autocomplete and correction suggestions have been helpful. For example, once one variable has data type changed, it can auto-correct the other spots of data type for change. However, its code block generation is sometimes confusing with wrong syntax, and for newbie like me who does not know much about Swift system would be little painful.
gemma4:4b - This model requires more than 10GB of memory, so would push the MBP to use swap and heats up the laptop quickly because of intensive GPU usage. So have not used it much. One time when a tricky Swift build error could not be resolved by other models, I intentionally switched to this, and it solved it. I guess this is good choice for little more complexity. Will try running in PC.
qwen2.5-coder-7b - Originally by its name, expected coder to be at least capable of helping out on coding. But the prompts within some context cannot be correct all the time. Any big task like code generation of a function would be given inaccurate or incomplete suggestions. Simpler corrections and documentation queries are still good enough.
rafw007/qwen35-claude-coder:9b - Since qwen2.5-coder-7b often provides incomplete explanations for humans, I tried to upgrade a version to Qwen3.5. No official smaller quantitized version other than open-source post on Unsloth/Huggingface pages. Had one try with this one from rafw007: provided code suggestion but no wordy explanation.

More updates might come later.

DEV Community: Brian Gan

Trying Out Local Image to 3D Model Generation AI Models

Let AI Write the Mac OS App Without Learning Swift