Running LLMs Locally on Consumer Hardware — Part 9: Image Generation on AMD, and a Creative Capstone

Part 9 of a series. Previously: Part 8 — The QA Problem.

The team could research, write code, administer a machine, verify its own work, and adapt when blocked. One thing it could not do was make a picture. This final entry is about adding that last capability — which on AMD hardware is less trivial than it sounds — and about the capstone project that used it: an attempt by the team to write and illustrate its own animated story.

The setup, at a glance

Before the procedure, the shape of it — three machines, each given the job its hardware actually suits.

Machine	Hardware	What it runs	Memory used	Headroom
Main ("gatsby")	Ryzen 5950X · ~80 GB RAM · RX 6900XT, 16 GB VRAM	Coordinator (gpt-oss:20b) + SDXL image generation	coordinator ~14 GB RAM (GPU idle); SDXL ~15 GB VRAM at peak	~60 GB RAM free
Worker (.114)	Ryzen 5600G · 32 GB RAM · GTX 1060, 6 GB VRAM	Specialist workers (Gemma 4 26B)	~18 GB model, spilling across 6 GB VRAM + system RAM	~14 GB RAM free
Laptop (.247)	Ryzen 8840U · 32 GB DDR5 · Radeon 780M (iGPU, no dGPU)	SD 1.5 image node, 512 px	light; the iGPU shares system RAM	ample — but SDXL will not run here

Image generation runs on the main machine's 6900XT, and the card is free to host it because the coordinator barely touches the GPU: too large for 16 GB of VRAM, it runs on the CPU and system RAM and leaves the card idle during inference. The two tenants therefore contend for different resources — the coordinator for RAM, SDXL for VRAM — so the real constraint is how much RAM the coordinator leaves behind:

Main machine (~80 GB RAM, 16 GB VRAM)	RAM used	VRAM used	RAM free	Image generation
Coordinator gpt-oss:120b + SDXL	~77 GB	~15 GB	~3 GB	times out
Coordinator gpt-oss:20b + SDXL	~14 GB	~15 GB	~60 GB	runs

The VRAM column is identical in both rows — the GPU was never the bottleneck. Right-sizing the coordinator from the 120B down to the 20B (Part 5) is what opened the ~60 GB of headroom that lets generation and coordination run at once. (SDXL's ~15 GB is the peak during a generation; ComfyUI's /free drops it back to about 2.2 GB when idle.)

Getting ComfyUI onto an AMD card

The plan was ComfyUI with Stable Diffusion on the main machine's now-idle 6900XT. Getting there was a ROCm fight of the familiar kind. ComfyUI's backend needs torch.library.custom_op, which arrived in PyTorch 2.4; the ROCm wheel that installed first was older, so the first launch failed immediately:

AttributeError: module 'torch.library' has no attribute 'custom_op'

The fix was to install a newer ROCm build (torch 2.5.1+rocm6.2) explicitly, and to launch with HSA_OVERRIDE_GFX_VERSION=10.3.0 — the same RDNA2 override Ollama needed back in Part 1. With that, ComfyUI saw the card natively: Device: cuda:0 AMD Radeon RX 6900 XT : native, 16 GB of VRAM.

The next failure was more instructive than it looked. The first model tried needed separate diffusion, CLIP, and VAE files, and the VAE folder was empty — so the pipeline produced a raw 16-channel latent and handed it, undecoded, to the step that saves a PNG:

TypeError: Cannot handle this data type: (1, 1, 16)

That shape is a tell: a 16-channel "image" is an undecoded latent, which means the VAE is missing or unconnected. Rather than chase the multi-file setup, the project switched to SDXL base — a single checkpoint with the VAE baked in, one node to load it, and a well-trodden default workflow. It generated a clean 1024×1024 image in roughly 15 to 30 seconds. Reliability beat novelty, which has been the recurring verdict throughout.

A sharp capability envelope

The laptop from Part 2 — the "AI PC" that failed as an LLM worker — was tried as a second image node, and here the story turns almost redemptive: a machine useless for agents might still be useful for something. It was, but within strict limits. SDXL hung the 780M hard, every time, at the first sampling step:

UserWarning: Attempting to use hipBLASLt on an unsupported architecture!
HW Exception by GPU node-1 ... reason :GPU Hang
Fatal Python error: Aborted

No override or attention flag got past it; AMD's own math libraries simply do not fully support that integrated GPU. But dropped down to SD 1.5 at 512×512, the same machine worked, stably, at around 37–42 seconds an image. The envelope was sharp and worth recording: SDXL hangs on the 780M; SD 1.5 at 512px runs fine. The card that could not run an agent had found the one image job it could do — which is the whole "match the workload to the machine" theme of this series, stated one more time.

The image generator is a tool, not an agent

One design decision is worth calling out because it runs against the grain of everything else here. Every capability so far had been embodied as an agent — a profile with a role. Image generation was not. It was built as generate-image.py, a plain script that posts an SDXL workflow to ComfyUI's API, polls for completion, and saves the PNG. Any agent that needs a picture calls it the way it would call any command-line tool; it was simply added to the relevant workers' instructions so they knew it existed.

The point is that not everything should be an agent. A deterministic, single-purpose capability — like a control loop in Part 6 — belongs in code, invoked as a tool, not dressed up as a teammate with a personality. There was no "artist agent," and the system was simpler for it.

The capstone: an illustrated story

With pictures finally possible, the team was pointed at its most ambitious task: write and illustrate a fifty-scene animated story, then assemble it into a website. It was a genuine stress test, and it exercised nearly every lesson in the series at once.

The seams showed immediately. Asked to write all fifty scenes in a single delegation, the researcher timed out repeatedly and, in the end, delivered around twenty full scenes plus an outline for the rest — a direct replay of the "few rich delegations, not one enormous one" lesson. Generating fifty images in one call was likewise impossible, so the team did something genuinely resourceful: it wrote its own batch script and ground through the images in chunks, retrying the ranges that timed out.

START_INDEX=0  END_INDEX=10  python3 generate_images_batch.py   540.7s
START_INDEX=10 END_INDEX=50  python3 generate_images_batch.py   900.3s [error]
START_INDEX=25 END_INDEX=35  python3 generate_images_batch.py   900.6s [error]
START_INDEX=25 END_INDEX=30  python3 generate_images_batch.py   328.6s

There were timeouts and a few mistyped output paths along the way, but the chunking worked: all fifty images were produced. The final assembly was where it came undone — the page meant to tie the story and images together was generated hollow, a title and some styling with none of the content, which is precisely the failure Hans caught and correctly refused to pass in Part 8. The capstone, honestly, was a partial success: the parts were all there and real, and the thing that joined them was not yet right.

The series, in one sentence

That partial result is a fair summary of the whole project. A team of small models, running on consumer hardware that mostly already existed, can do real multi-step work — research, code, system administration, verification, and now illustration — imperfectly but genuinely, at almost no running cost. Getting there was not a matter of finding one clever model. It was a long accumulation of unglamorous lessons, nearly all of which showed up in this last project at once: match the model to the job; respect the memory budget; put deterministic work in code, not in a model; keep the slow machine in the role it is actually good at; make the checker test what matters; and, above all, build a system that tells you the truth about what it did. None of it required a frontier lab's resources. Most of it required only paying attention to what broke, and why.

Thanks for reading. The code, the demos, and the data behind the series live in the companion repository.