DEV Community

Cover image for Notes on 3D Modeling Training with GPT-6 AGI
Lian0123 (連永立)
Lian0123 (連永立)

Posted on

Notes on 3D Modeling Training with GPT-6 AGI

GPT-6 Aatra was recently released, and many people online have been using it to create models with fairly good results. So I decided to give it a try. The result was a little painful to look at, which led me to start experimenting with ways to improve it.

The Problem

First, these are the three-view images I fed into the AI, generated by the AI itself:

This is the result created by GPT-6 Aatra. It is better than the 5.6 model, but still far from satisfactory:

After trying many different approaches, all of which failed, I suddenly had an idea: could the model’s default understanding of character modeling be fundamentally flawed—almost like a person with a cognitive impairment? If we fed the AI a character-modeling tutorial video, could that remove the obstacle and allow it to create a beautiful model?

The idea was based on OpenAI’s Video Pre-Training theory. If AI could learn how to perform tasks by watching YouTube videos, like in the Minecraft diamond-mining competition back then, then perhaps videos could help AI fully understand what it was supposed to do.

For a model with AGI-level capabilities, could we let it watch videos and use what it learned, together with reasoning, to make up for its lack of understanding? And since it is a model approaching AGI, could it learn an entire workflow after watching a video just once?

As it turns out, it can. Since I only have access to the Plus plan, I could only let it watch two short videos. Even so, it improved the model to the point where it looked like the handiwork of a temporary worker:

Overall, the result was fairly good. The rigging was still incorrect, but considering that I only used Plus and consumed just 60–70% of my weekly token quota, the finished model was already very good. At least, it looked like a person:

I’ve uploaded the files to GitHub. If you’re interested, take a look:

https://github.com/Lian0123/gpt-6-ai-3d-model-build-demo

Optimization

Since GPT-6 can do this, could I distill the process into a skill and make it possible for lower-end models to understand and reproduce it—achieving “token financial freedom”?

Reality dealt me a heavy blow. The skill was successfully created, and the three-view images were normal at first:

But the result was still a disaster. Although the body modeling was relatively accurate, the face was still a failure:

This seems like a very worthwhile topic to investigate. If I get any new results, I’ll update this article.

Further Thoughts

If we can rapidly train AGI through videos, could this method also make it possible to handle tasks such as special-effects production, video editing, and even high-end artistic creation?


The original article was written in Traditional Chinese, and the Japanese and English versions were translated using ChatGPT.

Top comments (0)