I Thought Better Models Would Fix AI Image Editing. I Was Wrong.

For a while, I assumed the biggest problem in AI image editing was obvious:

the model just wasn’t good enough yet.

If the output changed a face too much, use a better model. If the product logo became distorted, use a better model. If the lighting looked strange, wait for the next model release.

That explanation is comforting because it turns every failure into a model-quality problem.

But while working on an AI photo editing workflow, I kept running into a different conclusion:

the harder problem is expressing what is allowed to change — and what is not.

That sounds like a UX detail. I now think it is the core abstraction of AI image editing.

---

Generation and editing are not the same task

Text-to-image generation is wonderfully forgiving.

If I ask for:

A cinematic photo of a red sneaker on a concrete pedestal, soft studio light, dark gray background.

there are thousands of outputs that could reasonably satisfy me.

Editing is different.

Imagine I already have a product photo. I want to keep the exact shoe, its proportions, material, logo, color, and camera angle — but replace the background with a cleaner studio scene.

Now the request is not simply:

Generate a better product photo.

It is closer to:

Change the environment, but preserve the product identity.

That second half is where a surprising amount of the difficulty lives.

You can see the difference clearly in a before/after product edit:

The user is not asking the system to be creative everywhere. They are granting creativity inside a boundary.

That led me to a simple mental model:

// Conceptual model — not an API contract
const editRequest = {
  source: originalImage,
  change: "Replace the background with a clean studio setup",
  preserve: [
    "product shape",
    "logo",
    "material",
    "color",
    "camera angle"
  ]
}

The more I thought about it, the more this looked less like prompting and more like constraint management.

A more useful request object would also include explicit rejection conditions:

const editRequest = {
  source: originalImage,
  change: "Replace the background with a clean studio setup",
  preserve: {
    identity: ["product silhouette", "logo", "label"],
    geometry: ["camera angle", "scale", "perspective"],
    appearance: ["material", "product color"]
  },
  rejectIf: [
    "logo text changes",
    "silhouette changes",
    "product appears to float"
  ],
  review: ["logo at 100%", "contact shadow", "overall composition"]
}

That turns “preserve the product” from a vague instruction into something the product can review, log, and potentially expose in the UI.

---

Identity is not one field

A useful way to think about portrait preservation is that identity contains different classes of information. A current Phota Labs explainer on identity preservation separates relatively stable facial structure from dynamic expression and more transient features such as hair, makeup, glasses, or facial hair.

That distinction is useful for product design because an edit may intentionally change one class while protecting another.

For example, a professional-headshot request might allow:

• clothing to change;
• background to change;
• lighting to change moderately;

while treating these as hard constraints:

• facial structure;
• eye shape;
• jawline;
• nose and mouth proportions;
• recognizable expression characteristics.

The same idea applies to products. “Identity” might mean silhouette, label, logo, material, proportions, and a few distinctive details rather than a single generic similarity score.

This is why I prefer structured preservation fields over a single checkbox called “keep identity.”

---

“Change this” is only half of the prompt

Most people naturally describe the desired change first:

• remove the people in the background
• make the room Scandinavian
• turn daytime into night
• change the jacket to black
• add warm cinematic lighting

But for editing, the preservation requirements are often equally important:

• keep the face unchanged
• keep the original pose
• preserve the product logo
• do not move the furniture
• keep the camera perspective
• keep the typography intact

This is the pattern I now encourage in the interface and in prompts:

Change X. Preserve Y.

It is almost embarrassingly simple, but it produces a much clearer contract between the user and the model.

A room redesign is a good example. “Make this room more modern” is underspecified. Does the user want to replace the sofa? Move the windows? Change the floor plan?

A better request is something like:

Redesign the room in a warm minimalist style. Keep the room layout, windows, camera angle, and major furniture positions unchanged.

That is much closer to how a human art director would brief an editor.

---

When the constraint itself is ambiguous, ask before generating

There is one case where a longer prompt is not the right next step: the system does not yet know what the user has authorized it to change.

Suppose someone says:

Remove the person.

If two people are visible, silently choosing one is not prompt enhancement. It is guessing. The same problem appears with requests like “make it more professional” or “fix the label.” Those phrases may hide a real decision about target, operation, or edit scope.

Before expanding the instruction, I now prefer a tiny ambiguity gate:

1. Is the target unique? If not, ask which subject or region.
2. Is the operation explicit? If not, ask what should actually change.
3. Is the allowed scope clear? If not, ask whether the edit may affect only the target, the whole subject, or the whole image.
4. Do change and preserve rules conflict? If they do, surface the conflict instead of resolving it silently.

The important part is to ask the smallest question whose answer changes the edit. Do not turn a one-line image request into a questionnaire.

For example:

• “Remove the person” → “Which person — left or right?”
• “Make it more professional” → “Should I only clean up lighting, or can the background and crop change too?”
• “Fix the label” → “Should I restore the existing text, replace the wording, or only straighten the label?”

A useful decision rule is:

target ambiguous   -> ask which target
operation ambiguous -> ask what change
scope ambiguous    -> ask how broad the edit may be
otherwise          -> enhance inside the existing preserve contract

This separates clarification from enhancement. Enhancement can make a known instruction easier for a model to execute. Clarification is for the moments when the instruction itself is missing a decision.

---

Reference images are really another kind of constraint

Another thing I initially thought about too narrowly was reference images.

It is easy to treat them as “more prompt.” In practice, they are more useful as a visual contract.

A reference can answer questions that are annoying to describe in text:

• What shade of blue?
• What kind of lighting?
• How dense should the texture be?
• What does “minimalist” mean in this specific project?
• Which composition should the edit move toward?

In the workflow I’m building with ClipLumi, users can add up to three reference images alongside the main editing instruction.

The important part is not the number three. The important part is that the UI acknowledges something developers already know from multimodal systems:

language is not always the best representation of intent.

Sometimes the most precise instruction is another image.

---

Natural language does not eliminate the need for controls

There is a tempting product idea in generative AI:

If the model is smart enough, every setting should disappear.

I don’t think that is always true.

Some choices are not semantic. They are production constraints.

A user may know exactly what they want:

• 16:9 for a website hero
• 9:16 for a mobile story
• four variants instead of one
• a faster draft before spending time on quality
• a specific model because it behaves better for a certain task

Those are not failures of natural language. They are legitimate parameters.

So the current ClipLumi editor keeps explicit controls for things such as model selection, aspect ratio, output count, resolution, quality, fast mode, and style.

The design question becomes:

Which choices belong in prose, and which choices deserve a control?

My current rule of thumb is:

• Use language for intent.
• Use references for visual direction.
• Use controls for production constraints.

That separation has been more useful to me than trying to turn everything into one giant prompt box.

---

The goal was not “Photoshop with AI buttons”

This also changed how I thought about the product itself.

I did not want to rebuild a traditional editor in the browser and then add an AI button to the toolbar.

Traditional image editors are powerful because they expose the mechanics directly: selections, masks, layers, blend modes, brushes, curves, transforms.

For many everyday edits, though, the user does not care about the mechanics.

They care about the outcome:

Remove those people, but keep the subject and the scene intact.

The whole point of a prompt-driven editor is that the user should not need to manually paint a mask just to communicate that intention.

That is why the product direction became less about “AI features inside an editor” and more about editing through constraints, references, and natural language.

---

Hard gates and soft preferences should not be mixed

Another architecture lesson is to separate hard gates from soft preferences.

A hard gate is something that makes the output unusable if violated:

• the face no longer looks like the same person;
• the product logo changes;
• required text becomes incorrect;
• the product geometry changes materially.

A soft preference is something the user would like, but can trade off:

• slightly warmer lighting;
• more background blur;
• a stronger editorial mood;
• a different wall texture.

If both are flattened into one prompt, the model and the product layer have no concept of priority. The UI may show a beautiful result that failed the actual job.

A review pipeline should therefore evaluate hard gates first. Only after they pass should the product ask whether the output is aesthetically preferable.

That can be as simple as:

function reviewEdit(result, contract) {
  const hardFailures = checkHardConstraints(result, contract.preserve);
  if (hardFailures.length) {
    return { decision: "repair", hardFailures };
  }

  return {
    decision: "review-preferences",
    preferences: contract.preferences
  };
}

The code is conceptual, but the ordering matters.

---

Better models still matter — just not in the way I expected

None of this means model quality is irrelevant.

It matters a lot.

There are still cases where generative editing is frustrating:

• small text and logos can deform
• hands and fine geometry can change unexpectedly
• multiple edits in one request can interfere with one another
• preserving identity across aggressive transformations is difficult
• the same prompt can behave differently across models

A stronger model can reduce those failures.

But a better model cannot fully rescue a badly specified task.

If the interface gives the model permission to reinterpret the entire image when the user only wanted one small change, the product has already made the job harder than necessary.

This is why I increasingly think the next improvement in AI creative tools will not come only from larger models.

Current systems are improving at preservation and multi-round editing. OpenAI's Images 2.5 announcement explicitly calls out stronger subject fidelity, more precise editing, and better consistency across multiple edits. That is exactly the direction image editors need.

But even a model with better multi-turn consistency still needs the application to answer three product questions:

1. Which parts of the current image are already accepted?
2. Which failures are hard rejects rather than aesthetic preferences?
3. How small should the next repair be?

Some of the next improvement will therefore come from better ways to represent user intent and accepted state, not only from better generation quality.

---

A developer checklist before shipping an AI edit flow

Before I call an editing workflow production-ready, I now ask:

• Does the request model distinguish change from preserve?
• Can preservation constraints have priorities or hard-fail semantics?
• Can a reference image have a role instead of being an unlabeled attachment?
• Does the system know which image version is currently accepted?
• Can the user repair from the last accepted version instead of restarting?
• Is prompt enhancement observable and bypassable?
• Are model-specific capabilities derived from a capability map?
• Does the review flow check hard constraints before aesthetic preferences?
• Can operational logs explain which product-layer transformations occurred without storing unnecessary private image data?
• Does the next edit become narrower as more of the image is accepted?

If the answer to most of those is no, a better model may improve demos while the product still feels unreliable in repeated use.

---

The bigger question

So here is the question I’m curious about from other developers building with generative models:

Should future AI editors expose more model controls, or should they hide more of the model behind higher-level constraints?

In other words, are we moving toward:

1. increasingly sophisticated prompt boxes,
2. increasingly sophisticated traditional editors with AI features,
3. or a new kind of interface where users mainly define what may change, what must remain stable, and what references the system should follow?

I’m currently betting on the third direction.

That is the idea I’m exploring with ClipLumi: upload an image, describe the change, state what must remain intact, optionally add references, and keep enough explicit controls to make the output usable in a real workflow.

It is still imperfect, and that is exactly why I find the problem interesting.

If you were designing an AI image editor today, what would you expose to the user — and what would you hide?

I’d genuinely like to hear how other people here are thinking about this boundary.

Disclosure: I work on ClipLumi. I am using it as a concrete implementation example; the preservation-contract and hard-gate ideas above are general product patterns.

Comments

[Alex Shev]

That is a good distinction. Better models improve the ceiling, but editing is a workflow problem: references, masks, constraints, review loops, and knowing what must not change. Without that structure, the model keeps solving a different task each pass.