DEV Community: Efrain Garay

ComfyUI vs Forge on a 16 GB GPU: a tie on SDXL, ComfyUI twice as fast when the model does not fit

Efrain Garay — Sun, 18 Oct 2026 16:12:46 +0000

I use ComfyUI for images and video on a 16 GB card and had never put it side by side with Forge on the same machine. So I measured ComfyUI, Forge Neo and the original Forge on an RTX 4070 Ti SUPER with the same files, seeds and settings, diffusers as a baseline, 36 paired images per model and the decision rule written down before the first run.

Model	ComfyUI	Forge Neo	Original Forge	diffusers
SDXL (fits in 16 GB)	6.71 s	6.71 s	6.76 s	6.85 s
Z-Image Turbo bf16 (does not fit)	6.75 s	13.39 s	n/a	15.81 s
Qwen-Image fp8 (does not fit)	75.7 s	96.0 s	n/a	n/a

Median seconds per image, 36 images each.

While the model fits, they tie. With SDXL the three tools land within 1 % of each other.
When it does not fit, ComfyUI wins. Twice as fast with Z-Image Turbo, and 21 % less time per image with Qwen-Image fp8. Paired bootstrap intervals exclude a tie in both cases.
The logs point to memory management. ComfyUI v0.41 reports dynamic VRAM loading and «0 models unloaded»; Forge Neo logs three «Moving model(s)» lines per image, which add up to a median of 7.2 s with Z-Image, close to the 6.6 s gap.
Same seed, nearly the same image for ComfyUI and Forge Neo. The original Forge has no engine for Z-Image or Qwen-Image; its last commit on main is from June 2025.
Wan 2.2 A14B, one clip per tool as a curiosity: 877 s against 932 s, not enough to call a winner.

The post also covers what broke before measuring (a ComfyUI cache that faked a 0.05 s first image, a Forge Neo refiner switch that counted noise instead of steps) and the threats to validity.

Read the full measurements, the pre-registered sheet and the scripts → https://efraingaray.com/en/blog/comfyui-vs-forge-16gb/

NVFP4 vs FP8 vs GGUF on a 16 GB Ada GPU: NVFP4 decodes 2.3x faster and ties BF16 on GSM8K

Efrain Garay — Sun, 18 Oct 2026 01:56:53 +0000

When you type «nvfp4 vs» into Google it completes with «fp8», «gguf», «q4_k_m» and «bf16». The usual answer compares different models quantized by different people. On October 16 NVIDIA released Nemotron Labs Dory 1.0 4B in BF16, NVFP4, MXFP8 and GGUF at once, so I measured the same model in every format on an RTX 4070 Ti SUPER (Ada, 16 GB, no FP4 hardware), with the rules written down before the first run.

Format	Engine	Generate (tok/s, batch 1)	Read 2048-token prompt (tok/s)	Weights
BF16	vLLM	51.0	7,003	7.69 GiB
FP8 (per tensor, W8A8)	vLLM	78.5	10,118	4.67 GiB
MXFP8	vLLM	81.4	6,061	4.76 GiB
NVFP4	vLLM	116.8	6,978	3.28 GiB
GGUF Q5_K_M	llama.cpp	58.8	1,710	3.42 GiB (file)

Among the vLLM formats, generation follows weight size. NVFP4 was 2.3 times faster than BF16. On Ada it runs through a weight-only Marlin kernel: the weights stay small, the math stays in 16 bits.
Reading a long prompt, FP8 wins, because it is the only one of these formats whose multiplications Ada does in 8 bits.
Quality, full GSM8K (1319 questions, 8192-token cap): BF16 1249, FP8 1241, MXFP8 1250, NVFP4 1238. All three tie with BF16 under the ±3-point rule; their 95 % intervals stay within ±2 points.
Token by token it is another story. On fixed English and Spanish texts NVFP4 picks BF16's next token 89-90 % of the time, against 94-96 % for FP8 and MXFP8, while perplexity barely moves. MXFP8 is the most faithful to BF16.

The post also covers what broke before measuring (the llama.cpp converter rejects the public checkpoint, a prefix cache that inflated prefill, truncated reasoning being scored as answers) and a mistake the audit caught in my own first fidelity run.

Read the full measurements, the pre-registered sheet and the scripts → https://efraingaray.com/en/blog/nvfp4-vs-fp8-gguf-ada/

Kandinsky 6.0 on a 16 GB GPU: the official recipe runs out of memory, and only the 29B model passed my Spanish test

Efrain Garay — Wed, 14 Oct 2026 18:27:33 +0000

Kandinsky 6.0 generates five seconds of video with its audio, voices included, from a text prompt (MIT license). I wanted to know two things: does it run on a 16 GB card, and can it speak Spanish?

Memory. On a rented 16 GB RTX 4080, the README recipe (per-component CPU offload) ran out of memory with the distilled Lite model; the Qwen2.5-VL text encoder files alone add up to 15.45 GiB. An 8-bit text encoder and sequential offload failed too. What worked was a two-phase recipe: compute the prompt embeddings first, streaming the encoder through the GPU in groups of layers, then drop it and generate with a tiled video VAE. PyTorch allocated at most 11.24 GiB; a vertical clip took about 141 s; the process reached 45.9 GiB of system RAM.

Speech. Eight scenes with one Spanish line each, two seeds, scored with faster-whisper large-v3 against the requested line. The rule, written before running: median word error ≤ 0.20 and Spanish detected in ≥ 12 of 16 clips.

Model	Median Spanish WER	Clips with WER ≤ 0.20
Lite distilled (3B, 10 steps)	1.06	1 of 16
Lite undistilled (50 steps, guidance 5)	0.75	1 of 16
Pro distilled (29B, 10 steps)	0.15	9 of 16

Only Pro met the rule, and its process reached 120 GiB of RAM at roughly 12 to 23 minutes per clip (RTX 3090 with PyTorch memory capped at 15.57 GiB, transformer streamed block by block). The data is compatible with an advantage of the bigger model, but it cannot separate size from distillation.

The full write-up has the recipes, the component diagram, clips with what Whisper heard, the threats to validity (three external audits), and a demo: the small model's video with a replaced voice and the mouth redrawn by LatentSync 1.6.

Read the full measurements → https://efraingaray.com/en/blog/kandinsky6-video-audio-16gb/

Scripts, the pre-registration sheet and the raw JSONL: gist.

Qwen-Image 2.1 on 16 GB of VRAM, quantized to NF4: a real benchmark against FLUX

Efrain Garay — Sat, 10 Oct 2026 18:08:42 +0000

On a 16 GB RTX 4070 Ti SUPER, Qwen-Image-2.1-Turbo does not fit in bf16 (the pipeline is 32.5 GB). Quantized to NF4 with bitsandbytes it does, and I ran it against the FLUX.1-dev Q6_K + pixel-art LoRA that made my blog covers: same prompts, same seeds, same card, every image checked by eye.

What I measured

Speed: 8.4 s per image for Qwen NF4 (8 steps) against 61 s for FLUX + LoRA (30 steps), median, 1344×768. About 7×.
Text in the image: 13 of 18 requested texts exact for Qwen NF4, 7 for FLUX. Unquantized Qwen on a 48 GB A6000 got 17.
Cover prompts: Qwen NF4 drew the requested scene in 11 of 15 covers; FLUX + LoRA in none (6 almost, 9 missing the main action or object).
Memory: 11.9 GiB over the card's baseline for Qwen NF4 (nvidia-smi), 13.1 GiB for FLUX.

What broke on the way

An 8-bit (LLM.int8) transformer returns a near-uniform purple field. NF4 works.
Decoding the VAE in tiles (needed to fit) left faint vertical purple streaks. Fix: make the latents first, drop the transformer and the text encoder from the GPU, then decode the whole frame (0.77 s, 7.1 GiB peak).
An automatic reader (gemma3:4b) accepted misspelled text, and an external audit caught five of my own verdicts. Every number in the post comes from the corrected image-by-image review.

What I lose: the LoRA's atmosphere. Qwen follows the scene but on a flatter background. I switched anyway: every cover on the blog is now generated by Qwen NF4.

Read the full measurements, the side-by-side galleries and the code → https://efraingaray.com/en/blog/qwen-image-vs-flux/

Scripts, prompts and the per-image review: gist.

Strands Decider 2B vs LLM as a judge: a keyword regex tied the 2B

Efrain Garay — Wed, 07 Oct 2026 18:15:59 +0000

I wanted a cheap judge for my video pipeline: given a shot description, does it have a concrete subject, a visible action and an explicit camera decision, or is it generic? Strands Decider 2B is built for that kind of yes/no call, so I measured it.

75 shot descriptions: 63 captured from my pipeline (22 from production, 41 from a weak model running the same step) and 12 adversarial, with the three features annotated before running any judge.
With the choice primitive the 2B reaches AUC 0.91 in 34 ms per judgment. Claude Opus reaches 0.99. A regex that only looks for framing words reaches 0.90.
With its default primitive, noul, it caught none of the 31 generic shots at the 0.5 threshold. The v21 checkpoint does not improve it.
Repeated scenes in a shot list, my real failure, are the easy part: the 2B, Opus and Haiku got 13 of 13 lists right.
Shots, labels, every judge's output and the scripts are public: a CC-BY-4.0 dataset and a gist that reproduces every figure.

Read the full measurements → https://efraingaray.com/en/blog/juez-decision-2b/

Wild beats mold linking Rust 20 times out of 20, and in release the linker is no longer the bottleneck

Efrain Garay — Mon, 28 Sep 2026 01:01:46 +0000

I measured Wild 0.10.0, mold 2.42.1, rust-lld and GNU ld linking ripgrep and cargo.

Wild won all 20 repetitions of each project, by 1.9 ms and about 8 ms.
With either fast linker, the link is close to 1% of the rebuild — it stops being the bottleneck.
On the way I almost published false numbers; the post shows the trap and how I caught it.

Read the full measurements → https://efraingaray.com/en/blog/wild-vs-mold-rust/

TabPFN and TabICL against tuned XGBoost: the model that does not train won on fourteen tables out of fourteen

Efrain Garay — Mon, 28 Sep 2026 01:01:10 +0000

The claim behind TabPFN and TabICL: they predict on a table without ever training on it, and still beat tuned boosting.

Measured on 14 datasets from the Grinsztajn benchmark, same split and same clock for everyone.
The model that does not train won on 14 out of 14 against tuned XGBoost.
What it costs in latency and VRAM, and when I'd still reach for boosting — in the post.

Read the full measurements → https://efraingaray.com/en/blog/tabpfn-vs-xgboost/

Rust Coreutils 0.12 vs GNU 9.12: almost everything works the same, starting a process costs nearly 3x more

Efrain Garay — Mon, 28 Sep 2026 01:00:34 +0000

I measured Rust Coreutils (uutils) 0.12 against GNU coreutils 9.12 in containers with limits: the full GNU test suite, 157 commands with their output compared byte for byte, and process start-up.

uutils is slower in 91 of 157 cells.
It takes 2.8x longer to launch 5,000 processes.
mv between disks loses the timestamps.

Where each one wins, the full command matrix, and whether it's ready to replace GNU on a real box — in the write-up.

Read the full measurements → https://efraingaray.com/en/blog/rust-coreutils-contra-gnu/

Your architecture diagram is lying: I tested both ways of writing it as code

Efrain Garay — Mon, 28 Sep 2026 00:59:58 +0000

Most architecture diagrams in production are out of date — not through carelessness, but because they live in a PNG someone drew once while the code kept changing. Nobody opens a visual editor to move a box when they ship a microservice at 11pm.

A closed C4 web editor made the rounds on Habr this week: no repo, nothing to self-host, nothing to measure. So I tested the two alternatives you can run yourself, on a real system.

Structurizr CLI — generates all three C4 levels (context, containers, components) in 1 second from a single .dsl file.
C4-PlantUML — reaches the same result with nothing to install, at the cost of tripling the size of the model you maintain.

The full write-up has the exact commands, the four review gates this unlocks (diff, PR review, versioning, CI), the side-by-side of both tools, and when I'd pick each.

Read the full measurements → https://efraingaray.com/en/blog/arquitectura-como-codigo/

Don't put getaddrinfo on your proxy's hot path

Efrain Garay — Thu, 10 Sep 2026 21:11:48 +0000

I set out to benchmark Pingora 0.9.0 — the Rust proxy library Cloudflare runs at its edge — against nginx, in the environment I actually care about: a small pod with a CPU limit, two vCPUs, everything in containers so no number leans on the host.

I wrote the simplest possible reverse proxy with Pingora, hit it, and got 21k requests/sec against nginx's 126k in the same setup. Six times slower.

That number was a lie, and the fault was mine.

Six times is too much — Cloudflare would not replace nginx with something 6x slower. So I isolated it. Not CPU throttling (nr_throttled was zero). Not thread oversubscription (exactly two workers). Not connection reuse (both held ~130 upstream connections). The clue was latency: 4.78 ms in the container, 0.45 ms on the host with the backend at 127.0.0.1. The only thing that changed was how I named the backend.

My upstream_peer built the destination on every request:

// (&str, u16) → to_socket_addrs() → getaddrinfo, blocking, on the tokio worker
let peer = HttpPeer::new(("backend", 80), false, String::new());

HttpPeer::new over a (&str, u16) calls getaddrinfo synchronously, inside the tokio worker thread. In a container, every request fired a DNS query to Docker's resolver and blocked the thread. On the host, an IP literal is just a parse — no syscall — so it barely showed.

The fix is one line: resolve once at startup, keep the SocketAddr, pass that on the hot path.

let addr = ("backend", 80).to_socket_addrs()?.next().unwrap();
// ...on every request, no lookup:
Ok(Box::new(HttpPeer::new(self.addr, false, String::new())))

Pingora went from 21k to 89k req/s in the same pod. The 6x was a getaddrinfo per request, not the framework.

The lesson isn't about Pingora. A microbenchmark punishes any clumsiness of whoever writes it, and a blocking name resolution hidden on the hot path disguises itself perfectly as "the framework is slow."

With the proxy written properly, the real gap in a 2 vCPU pod is 1.49x — and perf pins it down to the cycle: Pingora runs 1.7x more instructions per request. I also found Pingora's thread default is 1 (a whole idle core if you don't set it), and that in a CPU-limited pod, more threads than cores wrecks the p99 through CFS throttling.

I documented the whole thing — the container setup, the full concurrency matrix with repetitions, the perf/strace profile, the flamegraph, and the thread-throttling numbers — with live diagrams and a reel, here:

👉 The full write-up on efraingaray.com

If you benchmark a Rust proxy in a container: resolve upstreams by IP or once, and set threads equal to your CPU quota. Those two lines matter more than the framework you pick.

pg_anon caught 1 of my 8 PII columns. My schema isn't in English.

Efrain Garay — Sun, 06 Sep 2026 03:20:27 +0000

pg_anon found 1 of the 8 personal-data columns in my PostgreSQL database. The one it caught was email, and only because "email" is spelled the same in Spanish and English.

The other seven — nombre, apellido, telefono, direccion, fecha_nac, tarjeta_ult4 and rut — walked straight through, unmasked.

pg_anon is the open TantorLabs tool that masks personal data in PostgreSQL: it scans the database, flags the sensitive columns, and dumps a masked copy. Like pg_dump, but covering the sensitive parts on the way out. Exactly what you want before handing a colleague a copy of production. So I fed it a Chilean database and watched it miss almost everything — no error, no warning. It finished successfully and handed me a dump with names and national IDs still in cleartext.

What the scan actually does

Two filters, in order. First it reads each column's name against a set of regexes (^email$, ^phone$, ^ssn$…). To the columns left over, it opens the data and tries patterns on the value (an email's @, a card's 16 digits). Whatever no filter catches passes through.

The rules in the demo meta-dict it ships with are written for English schemas. Mine aren't.

1 of 8

Column	Holds	Stock rules
`email`	email	✅
`nombre`	first name	❌
`apellido`	surname	❌
`telefono`	phone	❌
`direccion`	address	❌
`fecha_nac`	birth date	❌
`tarjeta_ult4`	card digits	❌
`rut`	national ID	❌

email got caught by name (it's an English word) and confirmed by its @. Everything else has a Spanish name no stock rule looks for.

The rut is the clearest miss. A RUT looks like 7917183-2: seven or eight digits, a dash, and a mod-11 check digit that can be the letter K. The only national-ID rule pg_anon ships with is ssn. It has no idea what a RUT is — and it won't know a cpf (Brazil), dni (Spain, Argentina), curp (Mexico), nif (Portugal) or aadhaar (India) either. If your schema isn't American, the defaults miss your most sensitive column.

The fix: a few lines of Spanish

You teach it. Column names in your language, plus a content regex for the ID so it's caught even in a column not named rut:

{
  "field": {
    "rules": ["^nombre$", "^apellido$", "^email$", "^telefono$",
              "^rut$", "^direccion$", "^fecha_nac$", "^tarjeta_ult4$"],
  },
  "data_regex": {
    "rules": [
      r"[\w.-]+@[\w-]+\.\w+",   # email
      r"^(\+?56)?9\d{8}$",     # Chilean mobile
      r"^\d{5,8}-[\dkK]$",      # RUT with check digit
    ],
  },
}

With that, recall went from 1 of 8 to 8 of 8, no false positives — it still left the non-personal columns alone. The point isn't "it went up when I wrote the rules" (obviously). It's how cheap the fix was, and that a masking tool ships blind to any schema that isn't English.

I left the full Spanish meta-dict (with the RUT regex) in a public repo so nobody starts from scratch: pg-anon-rules-es.

This is the short version. The full write-up covers what didn't fit: whether foreign keys survive the restore, the trap where a blanket SHA-256 keeps uniqueness but produces "emails" with no @, and how much slower it runs than a plain pg_dump. Numbers, diagrams and a 47-second breakdown: pg_anon found my database's emails, but not the RUT.

What does the national ID look like where you are — and would a stock masking tool catch it? Drop the format (or the regex) in the comments and I'll add it to the repo.

I translated DOOM from C to Rust in 23 seconds, then used AI for what it is good at

Efrain Garay — Fri, 21 Aug 2026 21:58:13 +0000

I needed to move a C program to Rust. My first instinct was the easy one: hand it to a model, chunk by chunk, and review the output.

Then I checked whether a tool already existed. c2rust, from Immunant, does exactly this. On the 73,095 lines of doomgeneric it took 23.2 seconds.

The interesting part is not the speed. It is that the model still ended up doing real work, just nowhere near the translation.

Cost was not the argument, and pretending otherwise would be selling smoke

I measured the token cost of translating DOOM with a model using the o200k_base tokenizer: 550,595 input tokens and 1,896,319 output. That is a few dollars. It is not a reason.

What actually decides is three other things:

Reproducible. Same input, same output file, byte for byte, every run. You can diff it.
Systematic errors. The translation produced four bugs. Two were the same bug in 23 places. One class gets fixed once.
All of them failed loudly. Every one crashed at compile time or on the first frame. A scattered, silent error in one of 143,911 lines costs more to find than the whole translation.

How you prove a translation did not change the program

DOOM ships recorded demos. Every gameplay decision is a fixed-point calculation, so if one of them differs the playback desynchronizes and the tic count stops matching. That is the first gate, and it is free.

The second gate: I hashed every frame the engine hands to the display, in both builds. 11,113 frames, one differed.

I nearly published that one frame as c2rust's single error. The control run is what saved me: I ran the original C binary against itself. It differs from itself too, on the same pixel. Original DOOM reads one pixel of uninitialized memory, and the translation reproduced that faithfully.

If you only ever compare against one run, you will report your own reference's noise as the other side's bug.

The translated Rust is not safe Rust

Measuring lines inside unsafe blocks, the translated zlib puts 47% of the crate in unsafe, against 31% for zlib-rs, a rewrite done by people.

Counting occurrences of the word unsafe gives you the opposite answer, and it is a misleading metric: the translator wraps whole functions in a single block, so it scores better on a count while being worse in reality.

These are not comparable artifacts anyway. zlib-rs is a rewrite, with redesigned structures and the type system doing work. An automatic translation cannot do that. Its goal is to not change the program.

Where the model actually earned its place

Not translating. Generating detail that modulates on top of what the engine already computes.

Every texture goes through an upscaler, then gets stored as a one-channel map whose mean is exactly 128 per texel. The renderer keeps deciding color and lighting; the map only contributes variation within the texel:

color_final = color_doom × detail / 128

Because the mean is anchored at 128, the enhancement provably cannot shift the image. Averaged over any texel, it multiplies by 1.

And that same map is a height map, so its gradient gives per-pixel relief for free, with no extra bytes, baked at pack time.

That constraint, that the engine keeps deciding, is what made the result add instead of ruin.

Both versions are playable in your browser

The 1993 C build and the Rust translation are compiled to WebAssembly and running side by side, plus the enhanced renderer at 640×400 with per-pixel lighting.

The 60-second version of all of this, with the game running behind it:

The full write-up has the four translation bugs, the three portability ones, the memset that silently cleared half the screen after widening a struct field, and how it went from 7215 to 3009 realtics without rewriting the game:

I translated DOOM from C to Rust without writing a line, then used AI for what it is actually good at