Instruction-based image editing looks like a targeted operation. You hand a model an image, type "replace the cup with a bouquet", and get the image back with a bouquet in it. The mental model that suggests is a patch applied to a region.
That mental model is wrong, and the cost of it being wrong shows up in details you were not looking at.
I spent a week running edits against a single source image to map the behavior. The subject was an anime bike courier I generated for the purpose: mint green undercut, slate blue jacket with one orange chest stripe, a fingerless glove on one hand, three enamel pins, a red bandana on the other wrist, a coffee cup, and a chalkboard sign next to him reading OPEN. Deliberately full of small, countable details, because those are the things you can measure.
Nine runs. Eight instruction edits on the same model, one masked run for comparison.
The setup
The edits ran on PixAI with Tsubaki.3, its newest model. Worth noting for anyone reproducing this: the source image goes into a Base Image slot that accepts exactly one file, and once it is loaded, the aspect ratio control disappears from the panel. The edit inherits the frame of the input. All eight outputs came back at the source image's exact dimensions, which makes before and after comparison trivial.
One more setting matters. There is a prompt helper that rewrites or translates your input before it reaches the model. Leave it off if you want to attribute results to your own wording.
Discrete swaps land, structure does not
The clean successes were attribute and object level.
A hair color change from mint green to navy came back correct, with the haircut, the shaved side, the jacket, and the face intact. A hand-held object swap, coffee cup to a paper-wrapped bouquet, came back with the grip and the wrist position preserved, which is the interesting part, since hand and object are drawn together.
Structural requests failed. I asked three separate times for a limb to move: twice for the gloved hand to come off the handlebar and point at the sign, once for the arm holding the cup to lift toward his chin. Nothing moved in any of the three. My first hypothesis was frame cropping, since the first arm sat half outside the image. The third attempt used an arm fully inside the frame and produced the same non-result.
Background replacement landed halfway. Asking for a rainy night street produced rain streaks and warmly lit windows while the daylight on the character and the pavement stayed exactly as bright as before. The additive parts of the request arrived; the parts requiring a global relight did not.
So the boundary is between local and global, and it is sharper than I expected.
The preservation clause is a real control, with limits
Half of my runs carried a long clause listing what had to stay unchanged: face, hair, glove, strap, pins, bandana, sign text, background, lighting, framing, art style.
The two variants of the gesture instruction looked like this:
# variant A, bare
Lift his gloved hand off the handlebar so he is pointing at the chalkboard sign.
# variant B, same instruction plus the keep list
Lift his gloved hand off the handlebar so he is pointing at the chalkboard sign.
Keep everything else unchanged: his face, his mint green undercut and the shaved
side, the silver hoop in his ear, the slate blue windbreaker and its single orange
chest stripe, the fingerless glove on that same hand, his other hand bare, the
crossbody strap, the three enamel pins on his chest, the red bandana on his wrist,
the paper coffee cup he is holding, the bike, the text on the chalkboard sign, the
shopfront and awning behind him, the lighting, the framing, and the art style.
Neither run performed the requested change, so everything else in the diff is drift. Without the clause, the glove changed from black to brown and his mouth opened. With it, both held.
The clause narrows the drift. It does not close it. Two items proved immune in every run that carried them:
- The three enamel pins. Count preserved in all eight runs. Colors and designs different in all eight.
- The text on the chalkboard. Seven runs had a goal other than the sign. All seven damaged it. Six of those seven named the sign text explicitly in the keep list.
The failure modes on the sign are worth listing, because they are not subtle: OPEN became handwritten scribbles, then handwriting with a small drawn heart, then something close to Pittal, then ANLEKN, then COFFFE, then DIUDE, then Nami with an accent over it.
The same model renders text well when text is the target. Asking for the board to read CLOSED produced correct spelling, correct placement, and a convincing chalk texture. This yields the one operational rule I would put in a runbook: text is the last edit in the chain, because any later edit will roll it.
Where the mask wins, quantified
Inpainting means painting a mask over the region you want changed, so the tool knows the edit is confined to that area, then describing what belongs there. It costs you brush work, and it buys you a scope guarantee.
I ran the same cup-to-bouquet change through a mask and diffed both results against the original.
| Method | Share of the frame that changed |
|---|---|
| Instruction edit | ~17% |
| Masked inpaint | ~2%, nearly all inside the mask |
The masked run left the chalkboard reading OPEN and the pins in their original colors. It also produced a visibly different bouquet, tied with a ribbon rather than wrapped in paper, which follows from the Edit panel running its own model rather than Tsubaki.3.
What I would tell my past self
- One change per run. Every run is a full re-render, so each additional request is another chance for something unrelated to move.
- Write the keep list. It measurably narrows drift, and it will not eliminate it.
- Local and discrete for instructions, global for regeneration. Pose and setting belong to a new generation, not an edit.
- Text last.
- Branch from the original for each new edit rather than chaining. Two runs redraw the small stuff twice.
- Diff against the original, not against memory. The pins are the thing you will never notice by eye.
The broader point is that "edit" in these tools is a compatibility label for something that behaves like conditional regeneration. Treat the output as a new image that happens to look like the old one, and your review habits fall into place.
If you want to try the same tests on your own image, the model I used is here: PixAI
Top comments (0)