DEV Community: Rodion Kazennov

MPS backend out of memory on a Mac: what the numbers mean and how to set the watermark

Rodion Kazennov — Wed, 14 Oct 2026 00:01:59 +0000

When a PyTorch job runs out of memory on a Mac, the error ends with a piece of advice:

RuntimeError: MPS backend out of memory (MPS allocated: 18.50 GiB, other allocations: 448.00 KiB, max allowed: 18.72 GiB). Tried to allocate 512.00 MiB on shared pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

I wanted to know what following that advice does, so I filled MPS memory in 512 MiB steps on my 48 GB M5 Pro (macOS 26.5.2, PyTorch 2.14.0), once per setting, under a guard that stops the process when swap grows by 2 GB. Two things came out of it. On this Mac the default limit is higher than the RAM, so a job that outgrows memory slides into swap and never gets to this error. And lowering the limit takes two variables, not one.

Where "max allowed" comes from

max allowed is the limit: PYTORCH_MPS_HIGH_WATERMARK_RATIO times what torch.mps.recommended_max_memory() returns. On my Mac that is 37.44 GiB, and 0.5 × 37.44 = 18.72 GiB, the number in the error above. MPS allocated matched torch.mps.current_allocated_memory() in my log, the memory held by tensors. To see your own base number:

import torch
print(torch.mps.recommended_max_memory() / 2**30, "GiB")

Why a 48 GB Mac swaps first

PyTorch's documentation gives the default high ratio as 1.7. On my Mac that puts the limit at about 63.6 GiB, more memory than the machine has. This is what each setting did:

setting	what stopped the fill	tensors held	swap during the run
default (1.7)	the guard	38.5 GiB	6.1 → 12.0 GB
0.0	the guard	36.0 GiB	5.1 → 9.8 GB
1.0, low 1.0	the guard	34.5 GiB	4.6 → 8.7 GB
0.8, low 0.8	PyTorch, max allowed 29.95 GiB	29.5 GiB	no change
0.5, low 0.5	PyTorch, max allowed 18.72 GiB	18.5 GiB	no change

With the default, with 0.0 and with 1.0, swap started to grow once the process passed 33.5 to 36.2 GB, and PyTorch never raised: its limit was further away than the end of the RAM. So 0.0, the advice inside the error, changed nothing I could see here. The limit was already out of reach, and nothing in PyTorch stopped the run.

Setting it: two variables

Setting only the high ratio fails at the first MPS call:

$ PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.8 python fill.py
RuntimeError: invalid low watermark ratio 1.4

The low ratio defaults to 1.4 on Apple Silicon and may not be above the high one (unless the high one is 0.0, which turns the limit off). Set both:

PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.8 PYTORCH_MPS_LOW_WATERMARK_RATIO=0.8 python run.py

From Python, set them before the first tensor goes to mps. PyTorch reads them when MPS starts, not at import torch, and ignores changes made after that:

import os
os.environ["PYTORCH_MPS_HIGH_WATERMARK_RATIO"] = "0.8"
os.environ["PYTORCH_MPS_LOW_WATERMARK_RATIO"] = "0.8"
import torch

The documentation calls the low ratio a soft limit and says that "setting 0.0 disables adaptive commit and garbage collection". I set it equal to the high one and did not measure it on its own.

Picking a value

The limit should stay below the memory your Mac really has free, with your usual apps open. On mine 1.0 (37.44 GiB) was already too much, and 0.8 (29.95 GiB) gave a clean error with no swap. After a clean error you can lower the batch size or the resolution and try again; a run sliding into swap slows down the whole Mac first.

A trap when you test this

My first run filled the tensors with torch.ones. With the default setting it reached the 46 GiB cap I had set, with no error and no swap at all: macOS compresses pages of identical values almost to nothing. Model weights don't compress like that, so a memory test on a Mac needs random data.

When the model really doesn't fit

A clean error only tells you the model doesn't fit. For a pipeline of several models, loading only the models of the current stage does more than any ratio, which is why I wrote stageload. The fill script and the traces of every run above are in its repository, in bench/results/2026-10-13-mps-watermark.

PyTorch on a Mac: why memory looks freed after .to("cpu")

Rodion Kazennov — Tue, 13 Oct 2026 08:43:56 +0000

On a Mac, a tensor moved from mps to the CPU seems to disappear from memory. The process footprint, the number Activity Monitor shows in its Memory column, drops to almost nothing, and so does the MPS driver count. The copy is still there and still takes the same memory. This is the minimal example I put together for a diffusers discussion about offloading, with a 4 GiB tensor standing in for a model's weights (torch 2.14.0, macOS 26.5.2, M5 Pro with 48 GB):

import ctypes, os, time
import torch

# rusage_info_v2 from <libproc.h>; phys_footprint is the Memory column of Activity Monitor
FIELDS = ("user_time", "system_time", "pkg_idle_wkups", "interrupt_wkups", "pageins",
          "wired_size", "resident_size", "phys_footprint")
class RUsageInfoV2(ctypes.Structure):
    _fields_ = [("uuid", ctypes.c_uint8 * 16)] + [(f, ctypes.c_uint64) for f in FIELDS] \
        + [(f"rest{i}", ctypes.c_uint64) for i in range(10)]

libproc = ctypes.CDLL("/usr/lib/libproc.dylib")

def show(step):
    torch.mps.synchronize()
    info = RUsageInfoV2()
    libproc.proc_pid_rusage(os.getpid(), 2, ctypes.byref(info))
    print(f"{step:<14} footprint {info.phys_footprint / 2**30:4.1f} GiB, "
          f"mps driver {torch.mps.driver_allocated_memory() / 2**30:4.1f} GiB")

show("start")
w = torch.randn(2**30, device="mps")      # 4 GiB of float32, standing in for a model's weights
show("on mps")
w = w.to("cpu")                           # what model CPU offload does when a model's turn ends
torch.mps.empty_cache(); time.sleep(2)
show("moved to cpu")
float(w.sum())                            # read every page of the cpu copy
show("cpu copy read")

start          footprint  0.1 GiB, mps driver  0.0 GiB
on mps         footprint  4.2 GiB, mps driver  4.0 GiB
moved to cpu   footprint  0.1 GiB, mps driver  0.0 GiB
cpu copy read  footprint  4.2 GiB, mps driver  0.0 GiB

I ran it three times and got the same output each time.

What the numbers say

Right after the move, both counters are back near zero, but w holds all 4 GiB of its values. Once the CPU reads every page of the copy, the footprint is 4.2 GiB again. So a footprint taken between .to("cpu") and the next time the CPU touches those weights does not show them.

The reason the move frees nothing in the first place is the hardware. On Apple Silicon the CPU and the GPU use one pool of physical memory. On a PC, moving weights from the GPU to the CPU frees VRAM for the next model. On a Mac, the weights "on the CPU" sit in the same RAM the GPU needs.

Why it matters for model CPU offload

diffusers' enable_model_cpu_offload() keeps each model on the CPU and moves it to the GPU only while it runs. I ran Qwen-Image-2.1 at 1024 × 1024 that way on the same Mac (trace and write-up). The process footprint never went above 18.5 GB, which would fit on a 48 GB Mac with room to spare. Swap grew by 3.5 GB during denoising and had grown by 9.6 GB by the VAE decode, and my memory guard stopped the run before it wrote an image.

A chart of the footprint alone would have called that run a fit. The example above shows one way the footprint can stay low while the weights are still in memory. I didn't measure which pages went to swap in the offload run.

How to check your own run

Look at swap, not only at the footprint:

sysctl vm.swapusage             # compare "used" before and after the run
sysctl kern.memorystatus_level  # system-wide free memory, in percent

If swap grows while the job runs, it doesn't fit, whatever the footprint says. stageload (pip install stageload), a small library I wrote for this, records both in a trace every 0.5 s with MemoryMeter, and stageload guard stops a command once swap grows past a budget:

stageload guard --wait-free 40 --swap-budget 8G -- python generate.py

What does free the memory

For a pipeline that runs its models one after another, the fix is to load a model when its stage starts and release it when the stage ends, instead of parking it on the CPU. stageload's release(module) moves every parameter and buffer to PyTorch's meta device, collects garbage and empties the MPS cache; a tensor on meta has no storage, so the memory goes back even while other code still holds a reference to the module. With the text encoder (16.3 GB) and the transformer (13.3 GB) loaded one at a time this way, Qwen-Image-2.1 peaked at 19.0 GB on the same Mac, and swap did not grow in either of two runs. The full setup is in How to run Qwen-Image-2.1 on a 48 GB Mac without swap.

What I did not measure

The free memory of the whole system during the example: other jobs on this Mac move it by several GiB between runs, so I couldn't get a clean number.
Whether del plus torch.mps.empty_cache() gives the memory back as fully as moving to meta, and how quickly.
Other macOS and torch versions, and Macs with more memory.

Godot Movie Maker and screenshots on macOS without --headless

Rodion Kazennov — Mon, 12 Oct 2026 21:22:55 +0000

I wanted Godot to write screenshots and short videos from a script on my Mac, the way it does in CI on Linux. Godot 4.7.1 on macOS 26.5.2 can't do it in --headless, but it can with the real renderer and the window off the screen. One thing I haven't solved: the app still comes to the front while it runs.

Everything below ran on a 640 × 360 test project with one script:

extends Node2D

# Saves the 30th frame to the path given after "--" and quits.
var frames := 0

func _process(_delta: float) -> void:
    $Box.position.x = 80.0 + frames * 4.0
    frames += 1
    var args := OS.get_cmdline_user_args()
    if frames == 30 and args.size() > 0:
        await RenderingServer.frame_post_draw
        get_viewport().get_texture().get_image().save_png(args[0])
        get_tree().quit()

What --headless does with a capture

Godot.app/Contents/MacOS/Godot --headless --path . -- "$PWD/out.png"

The script reached frame 30 and then waited at await RenderingServer.frame_post_draw for good: the process was still running 15 seconds later, and nothing was saved. Reading the texture without the await fails at once:

ERROR: Parameter "t" is null.
   at: texture_2d_get (./servers/rendering/dummy/storage/texture_storage.h:110)

In --headless Godot runs a dummy renderer, so the viewport has no texture to read; the path in the error is the dummy renderer's texture storage. On Linux the usual way round is a virtual X display under a real renderer. Godot's macOS build draws through the macOS window server and Metal, so that trick doesn't carry over.

Render for real, off the screen

Keep the real renderer and keep the window out of the way. Window settings take effect when the window is created, before any script runs, so they go in an override.cfg next to project.godot for the length of the run:

[display]

window/size/no_focus=true
window/size/initial_position_type=0
window/size/initial_position=Vector2i(-6000, -6000)
window/size/always_on_top=false

Then start the app through open:

open -n -W -g -j -a Godot.app --stdout run.log --stderr run.log \
  --args --path "$PWD" -- "$PWD/out.png"
rm override.cfg

-n starts a new instance even if Godot is already open, -W waits until it quits, -g keeps it in the background, -j launches it hidden, and --stdout and --stderr bring its output back to a file. Without them open drops the output, because Godot isn't a child process of your shell. The frame came back at 640 × 360 in the scene's own colours: a mean RGB of 188, 117, 82, the orange background with the blue box on it.

Delete override.cfg when the run ends. Godot reads it on every launch, so a run interrupted before the rm leaves the next normal launch with a window that never gets keyboard focus.

Movie Maker: a PNG sequence, then an MP4

Movie Maker renders every frame at a fixed rate, so the video's timing doesn't depend on how fast the machine renders:

open -n -W -g -j -a Godot.app --stdout run.log --stderr run.log \
  --args --path "$PWD" --write-movie "$PWD/movie/f.png" --fixed-fps 24 --quit-after 48

48 frames at 24 FPS (movie length: 00:00:02:00), recorded in 00:00:01 (200% of real-time speed).

It wrote f00000000.png to f00000047.png, with the audio next to them as f.wav. ffmpeg turns the frames into a video:

ffmpeg -framerate 24 -i movie/f%08d.png -c:v libx264 -pix_fmt yuv420p -crf 18 film.mp4

The result was 48 frames at 24 fps, 2.0 seconds long.

Not solved: focus

no_focus keeps the window from taking keyboard focus, but Godot still makes itself the active app when it opens its window. I polled the frontmost app during two runs. Godot was in front in 1 of 21 samples during the single-frame run and in 4 samples in a row during the Movie Maker run, which took about a second, and the app I was working in came back to the front each time Godot quit. In September a copy of the app bundle with LSUIElement set to true didn't help either. How long Godot stays in front during a longer recording, I haven't measured.

Limits

Godot 4.7.1 on macOS 26.5.2, one Mac, the Forward+ renderer. I didn't try the Compatibility renderer.
The off-screen position was tested with one display.

ulimit doesn't cap memory on macOS: stop a job before it swaps

Rodion Kazennov — Mon, 12 Oct 2026 14:31:35 +0000

On Linux a job's memory can be capped with ulimit -v, systemd-run -p MemoryMax= or a cgroup, and a job that grows too big fails on its own instead of taking the machine down. macOS has no systemd and no cgroups, and ulimit refuses. This is macOS 26.5.2:

$ /bin/bash -c 'ulimit -v 1048576'
/bin/bash: line 0: ulimit: virtual memory: cannot modify limit: Invalid argument
$ /bin/bash -c 'ulimit -m 1048576'
/bin/bash: line 0: ulimit: max memory size: cannot modify limit: Invalid argument

resource.setrlimit(resource.RLIMIT_DATA, ...) in Python raises ValueError, and in the shell where ulimit -m had failed, a script went on to allocate and touch 3 GiB. What macOS does instead is compress memory and write it to swap. A model that doesn't fit doesn't fail: it slows down, and so does everything else on the Mac.

So the questions worth asking change: is there room before a job starts, and is swap growing while it runs?

Check swap while a job runs

sysctl vm.swapusage             # compare "used" before and after a run
sysctl kern.memorystatus_level  # system-wide free memory, in percent

The process footprint, the number Activity Monitor shows, doesn't count pages that are already in swap. In one run of Qwen-Image-2.1 with diffusers' model CPU offload on my 48 GB M5 Pro, the footprint never went above 18.5 GB while swap grew by 9.6 GB. A chart of the footprint alone would have called that run a fit.

Stop a job before it swaps

stageload, a small library I wrote for loading models one stage at a time, ships a guard for this (pip install stageload):

stageload guard --wait-free 40 --swap-budget 8G -- python generate.py

It waits until kern.memorystatus_level is at least 40 %, then starts the command. If swap grows more than 8 GB over what was in use at the start, or free memory stays under 10 % for two samples, it sends SIGTERM to the command's process group and, five seconds later, SIGKILL to whatever is left. It needs no root, and the command and its process group are the only processes it ever stops. Exit code 3 means the guard stopped the command and 4 that it never found room to start it; otherwise you get the command's own exit code.

That is how the eager Qwen-Image run ended: it got through all 20 steps, the VAE decode took the footprint to 43.6 GB, swap grew by 8 GB, and the guard stopped it before the Mac stalled.

Two heavy jobs at once

A guard decides alone. Two guards that find room at the same moment both start their jobs, and each job was sized for a Mac with nothing else running. On 5 October one of my runs started in the gap between two video jobs of another script, and the next video job took swap past its budget within 20 seconds.

stageload run puts the jobs in one queue for the machine. It takes the guard's flags. Here two jobs hold 3 and 2 GiB, started from two terminals:

stageload run --who images -- python3 alloc.py 3 10    # holds 3 GiB for 10 s
stageload run --who upscale -- python3 alloc.py 2 4    # holds 2 GiB for 4 s

$ stageload queue
ram     0 running  images               pid 32488   since 2026-10-12T14:08:56+00:00  python3 alloc.py 3 10
ram     1 waiting  upscale              pid 32499   since 2026-10-12T14:08:58+00:00  python3 alloc.py 2 4

The second job starts when the first one ends, and then the guard's checks run as usual:

14:08:58Z queue: waiting in ram (1 ahead: images)
14:09:08Z queue: start in ram after 10 s, swap 6.0 GB, stop above 14.0 GB or under 10% available twice

$ stageload queue --history 2
done   ram    exit 0    images               waited 0 s, ran 10 s, peak 3.0 GB  python3 alloc.py 3 10
done   ram    exit 0    upscale              waited 10 s, ran 4 s, peak 2.0 GB  python3 alloc.py 2 4

Details that mattered on a shared Mac:

Jobs join a lane, ram by default. Lanes don't wait for each other, so a job that waits on a remote API quota can go in a lane of its own (--lane api --wait-free 0) and not hold up the ones that need memory.
Turns are fair between owners. The owner who started least recently goes next, so a script that queues twenty jobs takes turns with the others instead of holding the lane. The owner is --owner, or the part of --who before its first dash.
Each run writes a memory trace of the command and every process under it, and one line to ~/.stageload/history.jsonl with the wait, the run time, the peak footprint and swap.
Engines started outside the queue go in ~/.stageload/busy.txt, one pattern per line. Every run in the ram lane waits until none of them has been seen for 20 seconds.

Limits

The guard and the meter read macOS counters (libproc, sysctl), so they only run on a Mac.
The guard stops a job; it doesn't make the job smaller. On the model side, loading one model per stage is what took Qwen-Image-2.1 from 43.6 GB to 19.0 GB on the same Mac: How to run Qwen-Image-2.1 on a 48 GB Mac without swap.
For a safety net across the whole machine, macos-oom-guard runs as root and kills the largest process before the kernel panics. The two can run together.

How to run Qwen-Image-2.1 on a 48 GB Mac without swap

Rodion Kazennov — Sun, 11 Oct 2026 06:47:33 +0000

Qwen-Image-2.1 runs in diffusers through QwenImage21Pipeline. In bfloat16 its text encoder takes 16.3 GB, its transformer 13.3 GB and its VAE 1.3 GB, so the loaded pipeline came to 31.4 GB on a 48 GB Mac, and decoding a 1024 × 1024 image pushed it 11 GB higher. On that Mac, eager loading (the usual setup, with all three models on the GPU) and diffusers' model CPU offload both went into swap and never wrote an image. Staged loading, with one of the two large models in memory at a time, finished without swap.

The runs used an Apple M5 Pro with 48 GB, torch 2.14.0 and diffusers 0.41.0.dev0 from main. QwenImage21Pipeline also ships in the diffusers 0.41.0 release, which I did not run. Every run made one 1024 × 1024 image in 20 steps with seed 7, under a memory guard that stopped the run once swap had grown by 8 GB.

Eager loading goes into swap in the decode

The usual code loads all three models onto the GPU and keeps them there:

import torch
from diffusers import QwenImage21Pipeline

pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
).to("mps")
image = pipe(prompt=prompt, width=1024, height=1024, num_inference_steps=20).images[0]

It got through encoding and all 20 denoising steps at 31 to 33 GB. The VAE decode then took the footprint from 32.5 to 43.6 GB and swap grew by 8 GB, so the guard stopped the run before it wrote an image. PyTorch raised no out-of-memory error before that; the Mac went into swap instead.

Why model CPU offload does not help on a Mac

The answer diffusers has built in for a pipeline that does not fit is model CPU offload: each model waits on the CPU and moves to the GPU only while it runs.

pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload(device="mps")

On a Mac the CPU and the GPU share one pool of memory, so a model that waits "on the CPU" still sits in the same RAM. In the one offload run, the process footprint never went above 18.5 GB, which looks like a fit. Swap grew anyway: by 3.5 GB during denoising and to 9.6 GB in the VAE decode, and the guard stopped the run where it had stopped eager. The footprint, the number Activity Monitor shows for a process, does not count pages that are already in swap, so it cannot show this.

Load one model per stage

The pipeline uses its models in turn: the text encoder only to encode the prompt, the transformer only in the denoising loop, and neither in the VAE decode. So the two large models never have to be in memory together. stageload keeps track of that: StagedModels loads a model the first time a stage asks for it and releases it when the pipeline enters a stage that does not list it.

from stageload import StagedModels, on_call

STAGES = {"encode": ["text_encoder"], "denoise": ["transformer"], "decode": []}
models = StagedModels({"text_encoder": text_encoder, "transformer": transformer}, stages=STAGES)

on_call(pipe, "encode_prompt", before=lambda *a, **k: models.enter("encode"))
on_call(pipe, "prepare_latents", before=lambda *a, **k: models.enter("denoise"))
on_call(pipe, "_unpack_latents", before=lambda *a, **k: models.enter("decode"))

text_encoder and transformer are functions that load each model with from_pretrained(..., subfolder=...) and move it to mps. The pipeline itself is built with text_encoder=None, transformer=None and the VAE, which stays loaded. on_call runs models.enter(stage) just before the pipeline methods where each stage begins, so the pipeline's code stays as it is. One more piece is needed: the pipeline reads the transformer's config before denoising starts, so pipe.text_encoder and pipe.transformer are stand-ins that answer .config from the checkpoint and load the model on any other use. The whole script, with the stand-ins and a memory trace, is bench/qwen_image.py, and the library installs with pip install stageload.

Both staged runs finished, in 94 and 124 s. The footprint peaked at 19.0 GB while encoding, at 16.2 and 18.6 GB in the two runs' denoising, and at 14.4 GB in the decode. Swap did not grow in either run, and free memory never fell below 51 %. Loading the two models took about 13 s per run, and releasing the transformer before the decode another 4 to 5 s. Counted from the moment the transformer was in memory to the start of the decode, the 20 steps took 70 s in one staged run, 99 s in the other and 77.5 s for eager; the traces do not show why the two staged runs differ. Both produced the same image, bit for bit.

Check a run for swap

A run fits when swap does not grow while it runs, whatever the footprint says. Two commands show it:

sysctl vm.swapusage             # compare "used" before and after the run
sysctl kern.memorystatus_level  # system-wide free memory, in percent

stageload's MemoryMeter writes the footprint, swap and free memory to a JSONL trace every 0.5 s, and stageload guard starts a command when there is room for it and stops it once swap has grown past a budget:

stageload guard --wait-free 40 --swap-budget 8G -- python generate.py

What I did not measure

VAE tiling: pipe.vae.enable_tiling() decodes the image in tiles, and the decode is where the guard stopped both eager and offload. I did not try it, so I cannot say whether it lets eager finish on this Mac.
Sequential CPU offload, quantized weights, and Macs with less or more memory than 48 GB.
Whether staged loading gives the same image as eager: eager never wrote an image on this Mac.

The measurements in more detail are in Qwen-Image-2.1 one stage at a time and Model CPU offload on a 48 GB Mac, and the traces are in the stageload repository.

Model CPU offload on a 48 GB Mac: 18.5 GB footprint and 9.6 GB of swap

Rodion Kazennov — Sun, 11 Oct 2026 05:44:25 +0000

The second stageload note ran Qwen-Image-2.1 on a 48 GB Mac two ways. The memory guard stopped eager loading in the VAE decode, and staged loading finished. diffusers has a third way built in, enable_model_cpu_offload, which keeps each model on the CPU and moves it to the GPU only while it runs. On 8 October I ran it once, with the same prompt, size, steps, seed and guard limits as the runs of 6 October.

By the process footprint, offload looked like staged: it peaked at 18.5 GB, staged at 19.0 GB. Swap tells them apart. Encoding took 8 s and added no swap. During the 87 s of denoising, swap grew by 3.5 GB and available memory fell to 27 %. The VAE decode took swap growth to 9.6 GB and available memory down to 22 %, and the guard stopped the run 105 s in, before it wrote an image. Staged added no swap in either of its two runs and finished in 94 and 124 s.

The footprint cannot show this, because it does not count pages that are already in swap. On a Mac the CPU and the GPU share one pool of memory, so a model kept "on the CPU" sits in the same RAM as the model that is running. Judged by footprint alone, offload would sit next to staged. Judged by swap, it sits next to eager, which the guard stopped at the same point after 8 GB of new swap.

What I could not show

This is one run. A second would have pushed the machine into swap again for an outcome the guard had already decided. I did not measure which pages went to swap, and I did not try offload on a Mac with more memory.

The script is bench/qwen_image_offload.py, and the trace is in the repository. stageload installs with pip install stageload.

Loading a PyTorch pipeline one stage at a time: Pixal3D on a 48 GB Mac

Rodion Kazennov — Mon, 05 Oct 2026 13:00:36 +0000

On 4 October a Pixal3D run on my Mac went into its texture stage, and within ten seconds the process grew from 27 to 44 GB. Available memory fell to 6 % and swap grew by 12 GB, so the guard I run heavy jobs under stopped it. The Mac has 48 GB. Before I had that guard, two Pixal3D runs, one of them next to other heavy work, ended with the Mac rebooting.

Pixal3D turns one image into a textured 3D mesh, and Pixal3D-mac is its port to Apple Silicon. The 1024_cascade pipeline runs seven models in turn: a flow model and a decoder for the coarse structure, two flow models for the shape at 512 and 1024, a shape decoder, and a flow model and a decoder for the texture. Around them work four image feature extractors, a background remover and a camera estimator. Each stage needs one or two of the seven models, and the port keeps all of them in memory from the first second to the last.

Why the memory does not come back

On Apple Silicon the CPU and the GPU share one pool of memory. Pixal3D's low-VRAM mode, which is on by default, keeps idle models on the CPU and copies the active one to the GPU. On a PC with a separate graphics card, that saves video memory. On a Mac the CPU copy and the GPU copy sit in the same RAM, so the mode keeps every model resident and adds a second copy of the active one. The four feature extractors also load the same DINOv3 backbone, four times.

Loading one stage at a time

stageload is a small library I wrote for this. It swaps the pipeline's dictionary of models for one that loads a model the first time a stage asks for it, and that releases every model the next stage does not list. Releasing moves each tensor of the module to PyTorch's meta device. That frees the storage even while other code still holds a reference to the module, and calling the module afterwards raises an error that names it and the stage that released it, instead of failing somewhere inside PyTorch.

The port's own run() executes unchanged. stageload wraps four of the pipeline's methods to learn where each stage begins: background removal, the first image conditioning, the conditioning of each shape and texture pass, and the final decoding. The four feature extractors are built so that they share one backbone.

One part of this raises no error when it goes wrong. Pixal3D seeds once at the start of a run and then draws the noise of every stage from the CPU random generator. Building a model runs its random initialisation, which draws from that same generator. A model built lazily in the middle of the run would therefore change the noise of every stage after it, and with it the mesh. stageload saves the generators' state before each load and restores it afterwards: torch's CPU, MPS and CUDA generators, Python's random and NumPy's global one. On a stand-in pipeline that seeds and draws noise the way the port does, a test checks that staged and eager runs produce the same latents bit for bit, and a control that turns the restore off sees them drift apart.

Not only Pixal3D

Nothing in the core of stageload knows about Pixal3D. A pipeline needs three things from it: a function that builds each model on its own, a list of the models each stage uses, and a call where each stage begins. In code you own, that call is one line, models.enter("decode"). In someone else's code, like this port, on_call attaches it to a method that already runs at that point, so the pipeline itself stays as it is. A text-to-image pipeline has the same shape as Pixal3D: a text encoder, a denoiser and an image decoder run one after another. The memory meter and the guard work for any program on a Mac. Pixal3D is the only pipeline I have measured so far; the README shows the core on a small two-model pipeline, and that example runs as a test.

What it changed

On this M5 Pro with 48 GB, I ran the port's example image through 1024_cascade with seed 7 and 2048-pixel textures, two runs per mode, each in its own process under the guard. "Eager" is the port's own loading and "staged" is stageload. Eager runs do not get through the texture stage on this machine within the guard's limits, so the two eager runs stop where it begins. The texture number for eager comes from the run on 4 October that went into it.

stage	eager, peak footprint	staged, peak footprint
setup	20.4 GB	2.9 GB
camera	23.2 GB	6.7 GB
structure	22.2 GB	10.9 GB
shape_512	26.6 GB	11.4 to 11.5 GB
shape_1024	30.5 GB	13.1 GB
texture	stopped by the guard at 44.2 GB	30.1 GB
decode	not reached	17.9 to 18.4 GB
export	not reached	69.6 to 70.2 GB

In every stage both modes ran, staged held 11.3 to 17.5 GB less. The eight loads, the seven models plus the shape decoder a second time for decoding, read 13.2 GB in 24 s out of about ten minutes per run.

Most of the texture stage's 30 GB is working memory: the texture model itself is 2.6 GB. When decoding began, stageload released it together with the feature extractors and emptied the MPS allocator's cache, and within two seconds the footprint was down to 4.5 GB in one run and 8.0 GB in the other.

The highest staged number, about 70 GB, comes in the export, after every model weight has been released. It is the port's mesh processing (remeshing, UV unwrapping, baking), which stageload does not touch. The footprint is the number Activity Monitor shows, and it counts memory macOS has compressed, so it can be larger than the 48 GB of RAM.

What I could not show

The first is whether staged loading changes the result. Every run saved a hash and a copy of each sampler's output. The first latent, the coarse structure, is the same bit for bit in all four runs. From the 512 shape stage on, the port does not repeat itself on MPS even with the same seed: the two eager runs differ by up to 1.77 in the shape latent, and a staged run differs from an eager one by 1.24 to 1.86. So the comparison says any change from staged loading is no larger than the port's own variation, measured on that single pair of eager runs. The texture latent and the final mesh were not compared at all, because eager runs never reach them here.

Building on the CPU did not help. stageload builds each model directly on the GPU, and that changes one tensor the checkpoint does not cover: the rotary frequencies of the sparse attention, computed when a model is built, come out up to 6e-8 apart. To see whether that explained the differences, I ran a staged pass that built every model on the CPU and moved it to the GPU, the way the port does. Its latents differed from the eager runs by 1.84 and 1.12, the same order as every other pair. Loading took 82 s instead of 24 s, and three stages peaked 2.6 to 3.8 GB higher while two came out lower. I went back to building on the GPU.

My own summary had a bug that inflated one number. It took each stage's peak from the moment the stage was recorded as started, but releasing the previous stage's models takes up to 1.7 s. One memory sample in that gap put the decode peak of the CPU-built run at 29.5 GB; counted from the end of the releases, it is 16.7. A code review caught it. A stage's window now begins when the releases its start set off have finished.

The guard

The runs went through a second part of stageload, stageload guard, because other heavy jobs run on this Mac too. It starts one command only when enough memory is available and no process matching a list of heavy jobs has been seen for a while. While the command runs, it stops the command's whole process group if swap grows past a budget or available memory stays under 10 %. It needs no root, and the only processes it ever stops are the ones it started.

The wait for quiet came from a failure. A chain of video jobs leaves short gaps between them, a run started in one of those gaps, and the next video job took swap past the budget within 20 s. In the bench every run started only after three minutes with no heavy job in sight.

On another image

The same staged run on a character from a game I am making took 416 s and loaded the same 13.2 GB in 37 s. Its texture stage peaked at 30.1 GB, as on the example image. Here is the input and the mesh it produced, rendered in Blender:

The code, the traces of every run and the scripts that drew these charts are in the repository. The Pixal3D adapter is tested against one commit of Pixal3D-mac.

What 30 open-source projects ask of AI-assisted contributions

Rodion Kazennov — Sat, 03 Oct 2026 13:40:04 +0000

Between 27 September and 2 October I sent fixes, bug reports and comments to twelve open-source projects. A coding agent did much of the work: Claude Code reproduced the bugs, wrote the patches and tests, and drafted most of the texts. I chose each issue and read every text, and nothing went out from my account until I had approved it.

Along the way the projects kept telling me how they want that kind of work done. spec-kit wanted the agent, the model, its settings and the extent of its help named in every AI-generated comment. MLX's pull request template opens with a ticked line: "I understand it is strictly prohibited to use AI to write PR description". So on 2 October I read the contribution rules of 30 repositories: the ones I had sent work to, and 21 more that I had not.

The short version: 19 of the 30 have a rule about AI, and 14 of those 19 rules appeared in 2026. Most make the person who submits responsible for the change. Fifteen ask you to say that you used AI, and ten want a person, not a model, to write the description, the comments or the commit message. The commit trailer that coding agents add by default is required by some of these projects and banned by others.

What I read

The sample is not random. Nine of the 30 are projects I sent work to that week: MLX, spec-kit, huggingface_hub, codebase-memory-mcp, Deskflow, agent-framework, awslabs/mcp, kaggle-cli and avoid-ai-writing. The other 21 are projects in AI tooling, developer tools and infrastructure, among them Docling, garak, TypeScript, Rust, Node.js and Kubernetes. The three remaining projects I sent work to, copilot-cli, PartCrafter and ltx-2-mlx, are not in the 30, and I found no AI rule in any of them.

In each repository I read CONTRIBUTING, the pull request and issue templates, AGENTS.md and CLAUDE.md, the code of conduct, and the guides and wiki pages these files link to, at the default branch on 2 October 2026. Every quote below links to its line at the commit I read, so the link shows what the file said that day even after it changes. To date a rule, I searched the history of its file for the first commit that contains it.

Who has a rule

19 of the 30 have a rule about AI in contributions: MLX, spec-kit, TypeScript, huggingface_hub, transformers, trl, peft, LangChain, Strands Agents, MLflow, garak, codebase-memory-mcp, Deskflow, workers-sdk, pydantic-ai, Rust, CPython, Node.js and Kubernetes.

I found none in nine: accelerate, awslabs/mcp, kaggle-cli, Docling, BeeAI, ADK for Python, NeMo Agent Toolkit, VS Code and React. Two more I did not count. agent-framework only warns that AI has raised the number of contributions, so reviews take longer, and avoid-ai-writing checks the style of prose rather than how it was produced.

A rule usually asks for several things at once:

The second bar is my reading: 12 of the 19 say in some form that the person who submits is responsible for the change and must understand it. huggingface_hub does it in two sentences: "Using AI to help write code is fine. Submitting AI-generated slop you cannot explain is not." CPython expects authors "to be able to explain their proposed changes in their own words", and Strands Agents tells contributors "you are the author of your pull request, not your agent."

When they appeared

CPython was first, with a short page in its developer guide in October 2024. Four followed in 2025: spec-kit, LangChain, Kubernetes and MLflow. The other 14 appeared in 2026, seven of them since June. codebase-memory-mcp added its rule eleven days before I read it.

The files are still changing. In 16 of the 19, a file that holds the rule was edited in September or on the first two days of October, though not every one of those edits touched the AI text.

Say so

Disclosure is the most common rule, and the projects differ on how much of it they want. For Kubernetes one sentence is enough: its guide says that including "This PR was written in part with the assistance of generative AI" in the description "is sufficient." TypeScript closes the PR without it: "If your PR appears AI-authored and you do not include this disclosure, your PR will be closed without review." spec-kit wants the agent, the model, the settings and the extent named in the pull request, again in each AI-generated comment, and as an Assisted-by: trailer on each commit an agent wrote.

Two projects put the stakes in one line. codebase-memory-mcp: "Disclosure is never held against a contribution. Finding out later is." Deskflow: "Being dishonest about AI use in contributions will result in a ban from the project."

CPython asks for less than the rest: "Disclosure of the use of AI tools in the PR description is appreciated, while not required."

A person writes the words

Ten projects let a model help with the code but want a person to write the text. Rust is the strictest: "LLM-created PR descriptions are banned. LLM-created GitHub comments are banned." The same goes for commits: "Commit messages must be authored by you, not your LLM."

Kubernetes and Node.js draw the line at review. Kubernetes: "When responding to review comments, you must do so without relying on AI tools." Node.js: responses to feedback "must not be automated by AI tools."

Others build it into their templates. MLX has the line quoted above, and a matching one for issues. Strands Agents gives its pull request template a "Human Overview" section that "An AI agent MUST NOT fill out". LangChain's template warns: "If you paste a large clearly AI generated description here your PR may be IGNORED or CLOSED!"

No bulk, and ask first

Seven projects ban bulk or automated submissions. TypeScript describes the workflow it refuses: "workflows in which an operator points an autonomous agent at GitHub, has it generate patches across many unrelated issues, and forwards the output to us as pull requests." LangChain's guide gives a test that fits any contribution: "If the effort required to create a pull request is less than the effort required for maintainers to review it, that contribution should not be submitted."

Six want an approval before the pull request. In peft it is a maintainer's comment containing @peft-triage approved. In Rust a change created by an LLM must be pre-arranged, which means that "a reviewer has communicated ahead of time that they are willing to review an LLM-created PR."

Three projects turn away newcomers who use agents. transformers asks "that first-time contributors do not use code agents to create issues or PRs", and trl "will not review fully AI-generated PRs from first-time contributors." Node.js asks new contributors to "avoid using AI agents to interact with the project", and it bans AI fixes for issues labelled "good first issue", which "are meant to help new human contributors, not an AI, learn about the code base".

Rust also caps the share of this work: "If more than half of PRs merged in a 6-week window are LLM-created, we disallow merging new LLM-created PRs until we go back below 50%, with a minimum cooldown of 10 days."

The trailer they disagree on

Coding agents often sign their work. Claude Code, for one, adds a Co-Authored-By: Claude … trailer to each commit and a "Generated with Claude Code" line to pull requests unless told not to. Nine projects have a rule about such trailers, and the rules point in opposite directions:

The sources: MLflow, garak, spec-kit, Node.js, Kubernetes, Rust, pydantic-ai, Deskflow and codebase-memory-mcp.

Kubernetes: "Listing AI tooling as a co-author, co-signing commits using an AI tool, or using the assisted-by, co-developed or similar commit trailer is not allowed." pydantic-ai says it in the file Claude Code loads when a session starts: "Never add yourself (Claude) as a co-author on commits." Rust's guide lists both default lines among its bad examples of disclosure: 🤖 Generated with Claude Code and Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>.

So the default trailer is what MLflow asks for and a rule break in Kubernetes, Rust and pydantic-ai. No setting is right everywhere. An agent has to read a repository's rules before its first commit there, and so does the person who runs it.

Rules written for the agent

Some of these rules address the agent directly, in AGENTS.md or CLAUDE.md, the files coding agents read when they start work in a repository. What they tell it:

pydantic-ai: "you are the first line of defense against low-quality contributions and maintainer headaches".
MLX: do not "write PR descriptions and commit messages for the user", answer comments on the user's behalf, push or open a pull request.
Rust: stop, "name the prohibited category, and tell the user to author it."
transformers: warn people who are not yet contributors, "including the risk of being blocked."
spec-kit: ask the operator before a fourth open pull request in the repository, "including during autonomous operation".
Strands Agents, to agents that run on a schedule: "work at a pace humans can follow and respond to."

These rules work only if the agent reads the file and follows it.

What happened to mine

Kaggle/kaggle-cli: PR #1206, streaming kernels output files through the shared downloader. Merged the same day.
conorbronsdon/avoid-ai-writing: PRs #352, #361 and #362, two detector fixes and a test. All three merged, each within a day.
huggingface/huggingface_hub: issue #5065, --env-file lost its first variable when the file started with a UTF-8 BOM. A maintainer opened the fix, #5066, 52 minutes later, and it was merged within two hours of the report.
microsoft/agent-framework: PR #8963, data URIs with parameters. Closed after three hours as a duplicate: an older open PR, #8918, had the same fix.
ml-explore/mlx: PR #4610, Metal RMSNorm and LayerNorm with negative-stride weights. Closed after two and a half hours: "Thanks for the PR but this would not be the ideal fix."
DeusData/codebase-memory-mcp: issue #2454 and PR #2456, a misleading hint on empty search results. Labelled a high-priority bug, waiting for review.
dgrauet/ltx-2-mlx: PR #167, last and negative frame indices for image anchors. The maintainer asked for one fix before merging, and I sent it on 3 October.
github/spec-kit, microsoft/agent-framework and awslabs/mcp: a proposed scope on #4213 and plans for a fix on #8909 and #4705. Waiting for maintainers.
deskflow/deskflow: the root cause of #10068, a crash on macOS. Labelled and assigned, no reply yet.
github/copilot-cli: issue #5038, the grep tool ignores n without the dash. Labelled, no reply yet.
wgsxm/PartCrafter: PR #46, running on Apple Silicon. No reply yet.

Five of the twelve have an AI rule: MLX, spec-kit, huggingface_hub, codebase-memory-mcp and Deskflow. In huggingface_hub I only opened an issue, and its rule is about pull requests. In MLX I wrote the pull request description and the commit message myself, as its rules require, and the description says what the agent did. My spec-kit comment carries the full disclosure that spec-kit asks for, and in codebase-memory-mcp the issue, the pull request and each comment carry its one line. Two of my codebase-memory-mcp replies went out on 28 September without that line, a week after the project added the rule; I added it on 1 October. Deskflow asks for a disclosure in pull requests. My comment there, which the agent drafted, went out without one; I added the line on 3 October.

Neither closing comment mentions AI. agent-framework closed my PR because an older open one had the same fix, which I would have found if I had searched before writing mine. The MLX maintainer judged the approach.

What I changed

Before writing a fix, I search the open pull requests and the issue's timeline for the issue number. The duplicate came from skipping that.
In large projects I post a plan as a comment and open no pull request until a maintainer answers. agent-framework, awslabs/mcp and spec-kit are at that stage now.
One open pull request per project at a time.
The text follows each project's rule. If a project bans AI-written descriptions, as MLX does, I write them myself. spec-kit and codebase-memory-mcp get their disclosure in the form they ask for, and a project with no rule gets no disclosure line.
No trailers unless a project asks for one. My commits carry my name only.
I do not send this kind of work to projects whose rules it would not meet. Rust, Kubernetes and Node.js ban AI-written comments or replies to review, transformers, trl and Node.js ask newcomers not to use agents, and TypeScript refuses queue-driven work.

Limits

This is one account over six days. The 30 repositories are my choice and lean towards AI tooling, where rules are more likely. The kinds are my labels, and one rule often fits several. Everything was read on 2 October 2026, and in 16 of the 19 the rule files changed in the month before, so parts of this will be out of date soon. The pinned links show what each file said that day.

All 106 quotes, the counts and a script that checks every quote against its file are in nefayran/oss-ai-rules.

This note was drafted with Claude Code (Claude Opus 5.5), which also collected the rule files and checked every quote against them.

Where a code graph pays off: big repositories, small models

Rodion Kazennov — Thu, 01 Oct 2026 14:53:18 +0000

My last benchmark said a code graph did not make my coding agent cheaper. It was too small to say more than that: 24 questions, nothing bigger than 25k graph nodes, no Opus. So I ran it again at a size where the answer could change, and wrote the plan down first. The hypotheses, repositories, budget and analysis are in PLAN-v2.md, committed before any run.

The short version: a graph pays when the model would otherwise keep searching. Haiku on kubernetes and vscode is that case. Sonnet saves a little there and opus nothing, and on a small repository the graph costs sonnet more than plain grep.

The setup

The agent is GitHub Copilot CLI 1.0.90, run headless, one question per session, with two arms:

grep: Copilot's view, grep and glob tools and nothing else;
graph: the same three tools, plus cbm-lean (my thin wrapper around codebase-memory-mcp 0.9.0) and a routing rule at the top of the prompt: the graph for callers, callees and call chains, grep for exact strings.

Four public repositories, each pinned to a commit:

repository	language	graph nodes
full-stack-fastapi-template	Python, TypeScript	1,601
dub	TypeScript	25,023
kubernetes	Go	146,421
vscode	TypeScript	218,883

Each repository has 16 questions. Eight are structural: who calls a function, what it calls, the call path from a CLI command or an HTTP handler down to a named function, and which functions break if a signature changes. Eight are exact: a config value, where an environment variable is read, the line an error is raised on, the line a type is defined on. I checked every answer key by hand at the pinned commit, and an entry counts only as a whole word, so init_db does not pass for init.

Haiku 4.5 answered all 64 questions three times in each arm, and sonnet 5 twice. Opus 5.5 costs 15 premium requests a prompt in Copilot, so it answered twelve structural questions from kubernetes and vscode once. That is 664 runs and about 744 premium requests.

Cost is tokenEquiv, as in the first note: uncached input, plus 1.25 × cache writes, plus 0.1 × cache reads, summed over the model calls of a run.

Results

The graph's cost as a share of grep's: for each question the graph arm's median over repetitions divided by the grep arm's, then the median over questions, with a 95% bootstrap interval. Below 1 the graph is cheaper.

model	repository size	structural	exact
haiku 4.5	small (1.6k nodes)	0.88 (0.65–1.29)	0.92 (0.72–1.15)
haiku 4.5	medium (25k)	0.61 (0.11–1.50)	0.96 (0.75–1.37)
haiku 4.5	large (146k, 219k)	0.29 (0.11–0.70)	1.01 (0.94–1.28)
sonnet 5	small	1.51 (1.45–1.70)	1.42 (1.20–1.71)
sonnet 5	medium	0.94 (0.57–2.00)	1.14 (1.05–1.30)
sonnet 5	large	0.74 (0.62–0.92)	1.13 (1.01–1.29)
opus 5.5	large	0.97 (0.72–1.10)	–

Correct answers on the structural questions, graph against grep:

model	small	medium	large
haiku 4.5	13/24 against 19/24	20/24 against 22/24	38/47 against 33/48
sonnet 5	13/16 against 15/16	16/16 against 16/16	32/32 against 28/32
opus 5.5	–	–	12/12 against 12/12

Cost and accuracy together, as tokenEquiv per correct answer on the structural questions:

What holds up:

On kubernetes and vscode, haiku with the graph spent 0.29 of what it spent with grep on structural questions and was cheaper on 15 of 16 of them. It was also right more often, so per correct answer the gap is 47k against 209k tokenEquiv. This is the only result that survives the Holm correction across the 13 tests (p = 0.006).
Sonnet on the same questions: 0.74, cheaper on 12 of 16, and 32 of 32 correct against 28 of 32. Opus: 0.97, cheaper on 7 of 12, and every answer right in both arms.
On the small repository the graph never paid. Haiku spent about the same and got structural questions wrong more often. Sonnet spent 42 to 51% more.
On exact questions the graph made no measurable difference for haiku, and sonnet paid 13 to 42% more for it.
The graph's tool schemas and the routing rule add a fixed 1.56k prompt tokens to every model call for haiku and 2.07k for sonnet and opus, at every repository size.

Why the small model gains most

Count the model calls. With grep, haiku needed a median of 22 calls per structural answer on kubernetes and 20 on vscode, and one impact question took 114. With the graph it needed 5 and 4. Every call sends the conversation again, so a long search costs far more than its call count suggests: haiku's most expensive grep answer came to 471k tokenEquiv.

Sonnet's searches with grep were shorter, 8 calls against 4 with the graph. Opus took 4 or 5 calls either way, so the graph had nothing to cut, and its fixed overhead ate what little it saved.

My guess, which I did not test directly: a graph replaces search turns, so it pays when the model would otherwise take many of them, which means a weaker model on a larger repository. A strong model already searches efficiently with grep, and on a small repository nobody needs many turns.

Copilot bills premium requests per prompt, whatever the number of calls, so inside Copilot the graph saves no money. It saves time: haiku's structural answers on the large repositories took a median of 71 s with the graph and 157 s with grep. On the small repository the graph arm was slower.

Where each arm went wrong

A field called lines. Asked on which line HorizontalController is defined, the graph arm answered horizontal.go:43 in all five runs, haiku and sonnet alike. The struct starts on line 93 and is 43 lines long. The snippet that codebase-memory-mcp returns carries "start_line": 93, "end_line": 135 and "lines": 43, and the models read the last one as the line number. It happened again with LemonSqueezyClient (client.ts:129 for a class on line 59 that is 129 lines long), and haiku did the same with CursorsController. My first note blamed "line numbers from the graph" for that client.ts:129; this is where it came from. Sonnet with grep got all three right. cbm-lean now passes the field on as line_count.
Holes in the graph, trusted. codebase-memory-mcp 0.9.0 records no call edge for a bare call to a function imported with from x import y. Asked what recover_password calls, the graph arm answered "only get_user_by_email" in all five runs, haiku and sonnet alike, straight from the graph and without a single grep. The function also calls generate_password_reset_token, generate_reset_password_email and send_email, and the grep arm found all four every time.
A missing dash. Copilot's grep tool shows line numbers only when it gets "-n": true, and its view tool returns a file without them. Sonnet passed -n in 508 of its 571 grep calls and opus in all 53. Haiku passed n, without the dash, 920 times and -n 22 times out of 1,949. The tool ignores the unknown key, so haiku got no numbers, counted lines in the viewed file and missed by a few: for an error raised on line 112 it answered 105, 107, 108 and 113. All but one of haiku's 79 wrong answers to exact questions are a wrong line number.
Long searches that still fail. On the call chain from kubeadm token create down to encodeTokenSecretData, haiku with grep took 18 to 22 calls and got the path wrong all three times. It was wrong with the graph too. Sonnet was right in both arms.
Haiku tried to call bash three times in the grep arm. Copilot refused, since the tool does not exist there, so no run used a tool outside its arm.

What changed since the first note

The first note said not to expect token savings from a graph with Claude models on repositories up to 25k nodes. That still holds at those sizes. It missed what happens above them: on kubernetes and vscode the smaller models gain, and the smaller the model, the more.

What I would tell a team wiring a graph into an agent

Measure on your largest repository with your cheapest model first. That is where a graph earns its place.
With a strong model on a small or medium repository, leave it out. You pay for its schemas on every call and get nothing back.
Keep routing exact questions to grep. The graph never won them.
Do not let the model take line numbers or "this is the complete list" from graph output on trust. Drop or rename fields like lines before they reach the model.

Limits

One agent harness (Copilot CLI) and one graph (codebase-memory-mcp 0.9.0 behind my wrapper). Claude Code, another graph or a newer version could give other numbers.
Eight questions per cell on the small and medium repositories. With 13 tests under the Holm correction, a cell of eight cannot reach significance even if the graph wins every question, so only the large-repository haiku result is significant.
Opus answered twelve questions, once each.
tokenEquiv leaves output tokens out. Counting them would widen haiku's gap on the large repositories: its grep answers there produced a median of 7.0k output tokens against 0.9k with the graph.
One haiku run failed before Copilot opened a session and is left out. Three sonnet runs failed to fetch Copilot's model list before any model call and were run again. Both are recorded in the plan.

Reproduce

Everything is in code-graph-vs-grep: the plan written before the runs, the questions with the evidence for every answer key, the driver, all 664 runs with their answers, and the analysis.

node bench/run-copilot.mjs --model claude-haiku-4.5 --reps 3 --out results/v2/claude-haiku-4.5.json
python3 bench/stats-v2.py results/v2/*.json

Japan Life Hub: Net Salary Calculator & Housing Guide

Rodion Kazennov — Thu, 04 Dec 2025 00:00:00 +0000

Just shipped two new features for Japan Life Hub — a Net Salary Calculator with 2025 tax rates and a comprehensive Housing Guide. Both available in English, Japanese, Russian, and Hindi.

This article covers:

Why Year 1 vs Year 2 salary matters
2025 Japanese tax rates and deductions
Apartment rental costs breakdown
Key terminology for foreigners
Lessons learned building multilingual tools

The Year 1 Salary Trap

When I moved to Japan, my first paycheck was surprisingly high. Then Year 2 hit.

The catch: Japan's resident tax (住民税 juuminzei) is calculated on previous year's income and collected the following year. Your first year in Japan? Zero resident tax. Year 2? Roughly 10% of your gross salary disappears.

Nobody told me this. I built the calculator I wish I had.

Net Salary Calculator

The calculator shows both Year 1 and Year 2 take-home pay side by side.

2025 Tax Rates

Spent a day researching current rates from official sources. Here's what's deducted from your salary:

Social Insurance (Employee Share):

Item	Rate	Notes
Health Insurance	4.955%	Tokyo rate, March 2025
Long-term Care	0.795%	Ages 40-64 only
Pension	9.15%	Capped at ¥650k/month
Employment Insurance	0.55%	April 2025 update

Income Tax: Progressive 5-45% across 7 brackets, plus 2.1% reconstruction surcharge until 2037.

Resident Tax: ~10% (6% prefectural + 4% municipal), delayed one year.

Calculator Features

Employment type toggle: Employee vs Self-employed (different deductions)
Age selector: Long-term care insurance kicks in at 40
Dependents & spouse: Affects deductions for employees
Visual breakdown bar: See where your money goes
Salary presets: Junior ¥3M, Mid ¥5M, Senior ¥7M, Manager ¥10M
Share button: Copy summary to clipboard

The "What You Get Back" section explains what you're paying for — 70% healthcare coverage, pension from 65, unemployment benefits. Japanese social insurance is actually good value.

Housing Guide

Finding an apartment in Japan as a foreigner is confusing. Unique fees, guarantor requirements, Japanese-only listings.

The guide covers the entire process in 8 sections.

Key Terminology

Japanese rental vocabulary that confused me:

Japanese	Romaji	Meaning
敷金	shikikin	Deposit, 1-2 months, refundable
礼金	reikin	"Key money" gift to landlord, 0-2 months, gone forever
管理費	kanrihi	Monthly maintenance fee, ¥3,000-15,000
更新料	koushinryo	Renewal fee, ~1 month every 2 years
保証会社	hoshougaisha	Guarantor company, 30-100% of 1 month

Good news: less than 50% of renters now pay key money. Look for 礼金なし (reikin nashi) listings.

Initial Costs Example

For ¥80,000/month rent:

Cost	Amount	Refundable
First month rent	¥80,000	No
Deposit	¥80,000	Yes
Key money	¥80,000	No
Agency fee	¥88,000	No
Guarantor	¥40,000-80,000	No
Insurance	¥15,000	No
Lock change	¥15,000	No
Total	¥398,000-438,000

That's 5-6x monthly rent upfront. Budget accordingly.

Where to Search

The Housing Guide covers all major platforms — from Japanese-only sites like SUUMO with the largest listings, to English-friendly options like Real Estate Japan and GaijinPot, plus government UR Housing with no key money or guarantor requirements.

Pro tip: UR Housing has no reikin, no guarantor requirement, no renewal fees. Limited availability but worth checking.

Timing Matters

Avoid March-April. Everyone moves — new graduates, company transfers. Prices spike, competition fierce.

Best time: May-February. More negotiation room, landlords eager to fill vacant units.

What's Next

Japan Life Hub now has:

Initial Costs Calculator
Net Salary Calculator
Moving to Japan Guide
Housing Guide

Coming soon: Monthly Budget Planner, City Comparison Tool, First Year Timeline.

The goal is one place for everything you need to know about living in Japan — practical tools, not generic advice.

Try It

Built with Next.js, TypeScript, Tailwind CSS. All calculations run client-side, no data collected.

If you're moving to Japan or know someone who is — share the link.

Read on AllKeep →

Build a Chrome Extension with React, TypeScript and Vite

Rodion Kazennov — Wed, 03 Dec 2025 12:13:19 +0000

This article walks through building a Chrome extension from scratch using React, TypeScript and Vite. We'll build a Freelance Rate Calculator that helps developers check their market rate.

This article includes:

Setup React + Vite + TypeScript project for Chrome Extension
Configure manifest.json for Manifest V3
Build calculator UI with Tailwind CSS
Load and test extension locally
Submit to Chrome Web Store

Setup Project

Create new Vite project with React and TypeScript:

npm create vite@latest rate-calculator -- --template react-ts
cd rate-calculator
npm install

Add Tailwind CSS:

npm install -D tailwindcss postcss autoprefixer
npx tailwindcss init -p

Configure tailwind.config.js:

/** @type {import('tailwindcss').Config} */
export default {
  content: ["./index.html", "./src/**/*.{js,ts,jsx,tsx}"],
  theme: {
    extend: {},
  },
  plugins: [],
};

Add Tailwind directives to src/index.css:

@tailwind base;
@tailwind components;
@tailwind utilities;

Add Lucide icons:

npm install lucide-react

Add Manifest V3

Create public/manifest.json:

{
  "manifest_version": 3,
  "name": "Freelance Rate Calculator",
  "version": "1.0.0",
  "description": "Calculate your freelance rate based on tech stack, experience, and location",
  "action": {
    "default_popup": "index.html",
    "default_icon": {
      "16": "icons/icon16.png",
      "48": "icons/icon48.png",
      "128": "icons/icon128.png"
    }
  },
  "icons": {
    "16": "icons/icon16.png",
    "48": "icons/icon48.png",
    "128": "icons/icon128.png"
  }
}

Create public/icons/ folder and add extension icons in 16x16, 48x48, and 128x128 sizes.

Add Rate Data

Create src/data/rates.ts with tech stacks, experience levels, and locations:

export interface TechStack {
  id: string;
  name: string;
  baseRate: number;
  demand: "high" | "medium" | "low";
}

export interface ExperienceLevel {
  id: string;
  name: string;
  years: string;
  multiplier: number;
}

export interface Location {
  id: string;
  name: string;
  multiplier: number;
}

export const techStacks: TechStack[] = [
  { id: "genai", name: "GenAI / LLM", baseRate: 175, demand: "high" },
  { id: "rust", name: "Rust", baseRate: 135, demand: "high" },
  { id: "react", name: "React", baseRate: 90, demand: "high" },
  { id: "typescript", name: "TypeScript", baseRate: 92, demand: "high" },
  { id: "nodejs", name: "Node.js", baseRate: 85, demand: "high" },
  { id: "python", name: "Python", baseRate: 90, demand: "high" },
  // add more stacks...
];

export const experienceLevels: ExperienceLevel[] = [
  { id: "junior", name: "Junior", years: "0-2 years", multiplier: 0.55 },
  { id: "mid", name: "Mid-level", years: "3-5 years", multiplier: 1.0 },
  { id: "senior", name: "Senior", years: "5-8 years", multiplier: 1.45 },
  { id: "lead", name: "Lead / Architect", years: "8+ years", multiplier: 1.85 },
];

export const locations: Location[] = [
  { id: "usa", name: "United States", multiplier: 1.0 },
  { id: "uk", name: "United Kingdom", multiplier: 0.85 },
  { id: "germany", name: "Germany", multiplier: 0.82 },
  { id: "japan", name: "Japan", multiplier: 0.62 },
  { id: "india", name: "India", multiplier: 0.32 },
  // add more countries...
];

Rate calculation formula:

const getMarketRate = (
  stack: TechStack,
  exp: ExperienceLevel,
  loc: Location
) => {
  const base = stack.baseRate * exp.multiplier * loc.multiplier;
  return {
    low: Math.round(base * 0.85),
    mid: Math.round(base),
    high: Math.round(base * 1.15),
  };
};

Example: Senior React developer in Germany = 90 * 1.45 * 0.82 = $107/hr

Build Calculator UI

Update src/App.tsx with calculator form:

import { useState } from "react";
import { techStacks, experienceLevels, locations } from "./data/rates";
import type { TechStack, ExperienceLevel, Location } from "./data/rates";

function App() {
  const [stack, setStack] = useState<TechStack | null>(null);
  const [experience, setExperience] = useState<ExperienceLevel | null>(null);
  const [location, setLocation] = useState<Location | null>(null);
  const [myRate, setMyRate] = useState("");
  const [showResult, setShowResult] = useState(false);

  const getMarketRate = () => {
    if (!stack || !experience || !location) return null;
    const base = stack.baseRate * experience.multiplier * location.multiplier;
    return {
      low: Math.round(base * 0.85),
      mid: Math.round(base),
      high: Math.round(base * 1.15),
    };
  };

  const handleCheck = () => {
    if (stack && experience && location && myRate) {
      setShowResult(true);
    }
  };

  // render form or results based on showResult state
  return (
    <div className="w-[400px] h-[600px] p-4 bg-gray-900 text-white">
      {/* form inputs and results */}
    </div>
  );
}

export default App;

Set fixed popup dimensions in src/index.css:

body {
  width: 400px;
  height: 600px;
  margin: 0;
  overflow: hidden;
}

Input form with dropdowns for stack, experience, location:

Results screen showing market rate comparison:

Build Extension

Update vite.config.ts for extension build:

import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";

export default defineConfig({
  plugins: [react()],
  build: {
    outDir: "dist",
    rollupOptions: {
      input: "index.html",
    },
  },
});

Build the extension:

npm run build

This creates dist/ folder with your extension files.

Load Extension Locally

Open Chrome and navigate to chrome://extensions. Enable "Developer mode" in top right corner.

Click "Load unpacked" and select the dist/ folder.

Click the extension icon in toolbar to test the popup.

Submit to Chrome Web Store

Go to Chrome Web Store Developer Dashboard.

First time requires $5 one-time developer fee.

Prepare assets:

Extension icon: 128x128 PNG
Screenshot: 1280x800 or 640x400
Promotional tile: 440x280

Create ZIP of your dist/ folder:

cd dist && zip -r ../extension.zip . && cd ..

Upload ZIP, fill in description, add screenshots, submit for review.

Review takes 1-3 days. Common rejection reasons:

Keyword stuffing in description
Missing privacy policy (if using permissions)
Incorrect screenshots

After approval:

Conclusion

That's all. We built a Chrome extension with React, TypeScript and Vite, tested it locally, and published to Chrome Web Store.

Final stack:

React 18
TypeScript
Vite
Tailwind CSS
Manifest V3

Live extension: Freelance Rate Calculator

If you liked the article, follow me on Twitter @Nefayran for more #buildinpublic content.

Read on AllKeep →