There's a special kind of stubbornness that leads someone to say "I will generate photorealistic 3D humans, dress them with AI-designed clothing, give them personalities via an LLM, and drop them into a playable FPS/RPG hybrid — on a VPS with 6 vCPUs, 11 GB of RAM, and zero GPU."
That stubbornness is basically the whole architecture document for this project.
This post is a tour of two apps I've shipped inside my own CMS, CHARFORGE and 3D GAME ENGINE, why they're built the way they are, and the theory behind the tricks that make "no GPU" survivable instead of fatal.
The elevator pitch
CHARFORGE generates realistic 3D human characters entirely in the browser, gives them a name, backstory, and personality via a local LLM, dresses them using an AI garment generator running on the VPS, and can turn the 3D portrait into a photo-realistic image.
No OpenAI key, no Stability API bill, no vendor lock-in — every AI call happens on hardware I already pay for.
3D GAME ENGINE sits right underneath it: it takes any character you've forged, lets you assign it a role (enemy, boss, merchant, companion, whatever), drop it into one of 8 procedurally generated environments, and play the result in first or third person, WASD-and-mouse, right in the browser tab.
Both are "in production" as of this week, which in indie-VPS terms means: it's live, it mostly doesn't fall over, and I've personally watched five AI enemies die in a 25-second playtest with a 57% hit rate. Ship it.
Part 1: Why the browser does the 3D math, not the server
The single biggest architectural decision in both apps is this: the server never touches a GPU or does any 3D math.
All mesh deformation, skinning, and rendering happens client-side via Three.js. The VPS's job is to store JSON, serve static assets, and run two small local AI models.
Why? Because a 6-vCPU box with no GPU is a terrible renderer but a perfectly adequate file server and inference host for small models. Fighting that constraint instead of designing around it is how projects die. So:
CHARFORGE's human model is a MakeHuman (CC0) base mesh — 19,158 vertices, 163 bones — converted into a custom binary format and deformed by morph weights computed in JavaScript.
Skinning happens on the CPU, recomputed whenever a parameter changes.
3D GAME ENGINE bakes each character once — computing the final rest mesh with all morphs applied — and from then on only moves bones.
That single decision is what lets dozens of actors exist on screen simultaneously: you pay the expensive morph-and-rebuild cost exactly once per unique character, then GPU-skin (via Three.js SkinnedMesh) every clone afterward using SkeletonUtils.
That "bake once, animate cheaply" pattern is the whole trick behind making a browser feel like it's running a real game engine instead of a tech demo that chugs at 12 fps.
Part 2: A skeleton that speaks the same language everywhere
Both apps share a 163-bone skeleton built on Euler rotations in ZXY order, computed at each bone head as:
M = M_parent · T(h) · R · T(−h)
That's not decoration — it's the load-bearing piece of the whole cross-app integration. Because CHARFORGE and 3D GAME ENGINE agree on the exact same skeleton convention, a character forged in one app can walk, aim, and swing a sword in the other without a single retargeting step.
The engine code itself (charforge-engine.js, ~350 lines) is literally shared between both apps rather than duplicated — the game engine imports it directly.
This is the kind of thing that sounds obvious in a blog post and is not obvious at 2am when you're debugging why a cape is rotating around the wrong axis.
Getting the bone convention right once, up front, and refusing to let it drift between apps, is worth more than any individual optimization either app does.
Part 3: Two local AIs doing very different jobs, on the same box, with no GPU
This is the part I actually want to brag about. There are three local AI processes involved, and they're triaged by how "AI-shaped" the problem actually is:
Job Model Why this one
Character identity (name, backstory, personality, quote) Ollama running llama3.2:3b
Small enough to run on CPU with tolerable latency;
a name and three sentences of backstory don't need a frontier model
Garment fabric generation
Stable Diffusion (SD-Turbo), txt2img, CPU, ~512px, 3 steps Turbo variants trade quality for step count — 3 steps is the difference between "usable" and "the VPS catches fire"
Photorealistic character photo SD-Turbo, img2img, 4 steps Uses the 3D portrait as the starting latent instead of generating from noise, which is both faster and keeps facial identity consistent.
The theoretical point worth making here: you don't need a GPU if you're honest about which models actually need one.
A 3B parameter LLM and a "Turbo" distilled diffusion model at low step counts are specifically the tier of model designed to run acceptably on CPU.
The mistake most people make is trying to run a 70B model or a 50-step SDXL pipeline on a VPS and then concluding "local AI doesn't work without a GPU."
It's not that local AI needs a GPU — it's that some local AI does, and picking the wrong tier is a self-inflicted wound.
The practical cost of this choice is real, though: a garment takes ~35–40 seconds, a photo takes ~30–60 seconds, one job runs at a time, and the image worker eats about 4.4 GB of RAM — enough that it explicitly unloads Ollama when it needs the headroom. Text and image AI are, quite literally, taking turns.
Part 4: The queueing philosophy — never let AI generation hold a connection open
With nginx behind Cloudflare enforcing a 60-second timeout, and some jobs taking longer than that, the architecture never lets a single HTTP request wait for an AI job to finish. Every slow job goes:
POST the job → get a job ID immediately
Client polls GET /job/:id for status, progress, ETA
Server processes one job at a time in its own worker process
This is unglamorous, "everyone already knows this" system design — but it's exactly the pattern that gets skipped when people bolt AI onto an existing API and then wonder why Cloudflare is killing their requests. If your inference time is unpredictable, your HTTP layer should never assume it's fast.
Part 5: What the game engine adds on top — a small AI-adjacent behavior tree
3D GAME ENGINE isn't just "put the CHARFORGE model in a box." It adds a genuinely game-shaped layer:
8 AI behaviors per role — hunt, patrol, guard, follow, support, wander, idle, flee — with perception ticking every 0.2s across range, field of view, line of sight, hearing, and even night/fog visibility penalties.
A pathfinding* on a 1m grid with a hard cap of 4 path computations per frame, because letting 30 NPCs all replan simultaneously is how you turn a game into a slideshow.
Procedural everything — houses, trees, props, weather, sky — using InstancedMesh for repeated geometry, because instancing 200 identical crates is a rendering non-event, while 200 unique draw calls is not.
Combat math that's satisfyingly explicit: headshots multiply damage ×2.2, explosions fall off by distance, projectiles obey gravity and bounce.
None of this is exotic AI — it's classic game-AI (perception cones, flee thresholds, alert propagation), and that's the point.
The flashy AI (LLM identities, diffusion-generated clothing) lives in CHARFORGE; the load-bearing AI in the game engine is deliberately old-school and predictable, because 30 NPCs making decisions every frame is not where you want nondeterministic LLM latency showing up.
Part 6: The deployment discipline that makes any of this survivable
Here's the part that's less "cool AI" and more "how not to destroy your own VPS at 1am," which honestly might be the most reusable lesson in this whole post:
Every change is packaged as a diff against a known md5 of the live file, never a blind overwrite.
Backups happen before anything is touched — __backup_gameengine, __backup_editor, __backup_photo, and so on — checked with md5, and never stored inside the publicly served directory (learned that one the hard way, presumably).
Large payloads are chunked into ~50-line blocks, each verified with md5sum before being joined — because a naive 327-line paste got silently truncated by a terminal buffer, and that's a delightful bug to discover in production.
Installation is a single &&-chained command that stops at the first failure: dry-run patch → copy → real patch → syntax check (node --check, php -l) → done, or nothing.
Rollback commands are never put in the same code blocks as forward changes — because if you're pasting every block verbatim, you really don't want to accidentally paste the rollback.
None of this is AI. It's just operational paranoia, and it's the reason "Claude wrote code that patches a live production PHP CMS" wasn't a disaster.
What's actually still rough (the honest limits section)
In the spirit of not writing marketing copy: collisions are flat axis-aligned boxes, there's no mid-match save, no multiplayer, animation is procedural bone-rotation rather than motion-capture (so there's some foot sliding), and AI photo generation is deployed but not yet quality-tuned against real output.
The roadmap — quality settings, a placement editor, quest scripting, NPC dialogue actually generated live from the local LLM identity, WebSocket multiplayer — is long, and most of it is "depth," not "missing foundations."
Which, for a phase-1 shipped-on-a-VPS project, feels like the right kind of unfinished.
The takeaway, if you're building something similar
If there's a general lesson buried in two technical reports about a character generator and a browser FPS, it's this: local AI on modest hardware works when you match the model tier to the hardware tier, decouple slow inference from your request/response cycle, and put your engineering discipline into deployment safety rather than into pretending the constraints don't exist.
The LLM doesn't need to be huge. The diffusion model doesn't need 50 steps.
The server doesn't need a GPU. It needs a queue, a backup script, and the humility to bake a mesh once instead of recomputing it every frame.
Top comments (2)
the md5-before-patch thing is the real story here, not the 3d engine. most people just let ai overwrite and hope. did you ever hit a case where the file changed between your read and your patch, so the md5 check itself caught something you did not expect
Totally! 🎯
It actually happened a few times early on when manual edits conflicted with a pending patch, causing the MD5 check to instantly catch the mismatch during the dry-run.
That friction saves production. When automating with AI, the hardest part isn't generating good code—it's having a bulletproof safety check so an out-of-sync file doesn't break live at 1:00 AM.
Thanks for reading!