Instruction-based image editing has a failure mode that is easy to describe and hard to pin down: the requested change lands, and something you never mentioned comes back different. The face is a little off, an accessory is gone, the background warmed up by a few degrees.
The usual way to evaluate this is to look at the output and form an impression. I wanted numbers instead, so I set up something closer to a controlled experiment against PixAI's Tsubaki.3 and ran nine edits, the four main tests all starting from the same source image.
Setup
One source image, held constant. Same model, same mode, no style preset, no LoRA, prompt rewriting disabled, default negative prompt untouched, and a pinned seed. Once a base image is loaded the panel exposes no strength control, so there was nothing else to hold fixed.
The source was built as a test fixture rather than as a picture. Every identifying detail sits on one side only, which turns drift into something you can count:
left ear silver hoop right ear nothing
right wrist watch left wrist bare
right side stethoscope head left arm two bandage squares
table ginger tabby cat, white chest patch, one white front paw, red collar
The cat is the control. It sits in the middle of the frame, adjacent to everything I was about to modify, and exactly one of the nine instructions mentions it.
Measurement
For each result, I compared regions against the same regions of the source and took the mean per-pixel difference. Calibration matters more than the absolute values here:
| Region type | Value when untouched |
|---|---|
| Flat wall | 1 to 2 |
| Detailed area, table edge | 3 to 6 |
| A region the edit changed | tens |
Results
One instruction. Recolor a garment. Face 3.2, cat fur 5.0, cat eyes 3.8, watch 5.6, bandage 2.4, background 1.6. The change stayed inside the garment it named.
One structural instruction. Replace the outfit entirely, which forces the model to rebuild shape rather than repaint color. Stethoscope survived on top of the rebuilt torso. Both bandages survived. Cat at 4.5. Face at 10.1, against 3.2 in the recolor, with nothing in the prompt referring to the face.
Three instructions in one prompt. Cardigan color, held object swap, facial expression. All three landed. I then ran the same prompt twice more, once with a list of elements to preserve and once with that list plus the cat named by its markings. Whole-frame difference between the three variants: 1.4 to 2.6, which is background noise. The preservation list changed nothing measurable, because nothing in that run was under threat.
Four instructions plus an environment change. New location, new light source, new jacket, new held object, and a rendered sign reading ROOM 2. All four landed and the lettering came out clean. Color did not survive. The cat's fur went from an average RGB of (233, 181, 122) to (142, 149, 148), collapsing the channel spread from 111 to 7. Skin turned bluish grey and hair went near black, on a run whose preservation list named the face and hair explicitly, as most of the other runs did too.
The finding
Instruction count was the wrong variable. One instruction and four were carried with the same accuracy, and nothing was ignored across nine edits.
What scaled was reach. The disturbed area tracked how much of the picture the instruction obliged the model to reconstruct:
recolor a garment -> stops at the garment edge
rebuild an outfit -> reaches the face
change the light -> reaches every surface whose color depends on light
The preservation list follows the same curve. At one instruction it had nothing to save. At three it made no measurable difference. At four plus an environment change it named the face and hair and lost both. It costs nothing to include and it stops being sufficient at the point where you start needing it.
Chaining is worse than batching
The obvious instinct is to split a complex edit into steps. I ran the three-change instruction as three sequential edits, each starting from the previous result, and measured how broken up the flat wall behind the subject became:
| Route | Wall blockiness |
|---|---|
| Source | 0.36 |
| One prompt, three changes | 0.40 |
| Chained, step 1 | 0.43 |
| Chained, step 2 | 1.21 |
| Chained, step 3 | 7.35 |
By step three the breakup had spread onto the subject and the cat. Every pass re-encodes the whole image, and the artifacts compound in flat regions first. Instruction following survived the whole chain, and image quality did not.
I ran step two twice and got byte-identical files, so the result is reproducible on these settings. Whether a different seed avoids it is untested. Worth noting for anyone reaching for a retry: with the seed pinned, rerunning changes nothing at all. Freeing the seed or rewriting the instruction are the only two levers.
Practical takeaways
- Batch related changes into one instruction. Four coordinated changes came through intact.
- Expect a structural rebuild to move the face slightly. Check it against the original.
- Expect a lighting or environment change to rewrite color on skin, hair and fur, and expect a preservation list to fail there.
- Avoid chained passes on the same file. The quality cost is real and compounds.
- Pin your settings before a comparison run. Half of this analysis only works because every edit ran on identical parameters.
The full write-up with all the before and after images is on Medium. If you want to run the same experiment on your own file, Tsubaki.3 is on PixAI. Take something you already like, change one small thing, and go audit the corners of the frame.
Top comments (0)