Saviel Yamani

Posted on Aug 10

Retaining Natural Textures: How to Combine Image to Image AI and Blemish Removers

If you have spent any time working with generative AI for portrait editing, you have likely run into the "plastic skin" dilemma.

In the pursuit of clean, presentable skin, we often swing between two extremes. On one side, traditional manual editing tools can be painfully slow. On the other, modern generative models can easily over-smooth faces, stripping away the micro-textures—like pores, fine lines, and subtle peach fuzz—that make a human face look authentic.

To solve this, we do not need to choose between manual editing and artificial intelligence. Instead, we can build a hybrid workflow. By combining the precision of a targeted Blemish remover with the contextual synthesis of image to image ai, we can achieve clean, high-fidelity skin textures while preserving the subject’s true identity.

In this article, we will break down the underlying mechanics of both technologies, analyze why they often fail when used in isolation, and explore a structured, programmatic workflow to combine them.

1. The Core Technologies: Pixel-Patching vs. Generative Diffusion

Before we look at the hybrid workflow, it is helpful to understand how these two approaches process pixel data differently.

Traditional Blemish Remover (Pixel-level Reconstruction)

Traditional blemish removal tools (such as Photoshop’s Healing Brush, or bilateral filter algorithms in OpenCV) work by analyzing localized pixels.

[ Neighboring Texture ] ---> [ Source Texture Sampled ]
                                      |
                                      v
[ Blemish Area ] ---------> [ Color/Luminance Blended ] ---> [ Repaired Patch ]

When you click on a spot, the algorithm samples a clean area nearby, copies its high-frequency texture (the roughness), and blends it with the color and luminance of the target area.

The Strengths: It preserves the exact underlying structure of the original photo. It does not hallucinate new details or alter the subject's anatomy.
The Weaknesses: It is highly localized. If you attempt to use these algorithms over large areas (like resolving widespread skin redness or complex lighting transitions), the result often looks muddy, blotchy, or blurry because the algorithm lacks a semantic understanding of what "skin" is supposed to look like under specific lighting.

Image to Image AI (Semantic Synthesis)

An image to image ai model (such as Stable Diffusion using Img2Img or ControlNet) does not copy and paste pixels. It takes an input image, injects a controlled amount of Gaussian noise, and then uses a neural network to reconstruct a new image guided by a text prompt.

[ Input Image ] ---> [ Add Noise (Denoising Strength) ] ---> [ UNet Latent Processing + Text Prompt ] ---> [ Newly Synthesized Image ]

The Strengths: The AI has a deep, semantic understanding of human anatomy, light propagation, and material textures. It can generate incredibly realistic micro-textures (like individual skin pores) that match the ambient lighting.
The Weaknesses: AI models lack precise spatial memory unless heavily constrained. If you feed a portrait into an image-to-image pipeline with a high denoising setting, the AI will restructure the nose, shift the eyes, or alter the bone structure, effectively erasing the subject's identity.

2. Why Pure AI Struggles with Raw Blemishes

It is tempting to think we can just throw a raw, unedited portrait into an Image to image AI pipeline and expect it to clean up the blemishes. However, this often fails due to a concept known as feature amplification.

If an image contains a highly visible, high-contrast blemish (such as a dark spot or severe redness), the AI’s encoder interprets this as a significant structural feature.

If your Denoising Strength is set low (e.g., 0.15), the AI will try to preserve the original structure and may actually sharpen or retain the blemish, treating it as an essential detail.
If your Denoising Strength is set high (e.g., 0.5 or higher) to force the AI to overwrite the blemish, the AI will also overwrite the surrounding geometry—changing the shape of the cheeks, the mouth, or the eyes.

This is where the hybrid approach becomes necessary.

3. The Hybrid Workflow: Step-by-Step

By using a localized blemish remover before applying an image-to-image AI pass, we remove the high-contrast "anchors" that confuse the AI. This allows us to use a very low denoising strength during the AI phase, preserving the exact geometry of the face while leveraging the AI solely to reconstruct natural-looking skin textures.

Step 1: Localized Pre-processing (The Blemish Remover Pass)

First, use a standard spot-healing brush, bilateral filter, or an automated segmentation-based healing tool to clean up major spots, stray hairs, and stark imperfections.

Note: You do not need to make the skin look perfect at this stage. It is fine if the skin looks slightly flat or blurred where the spots were removed. The goal is simply to flatten the contrast of the blemishes so they match the surrounding skin tone.

Step 2: Generating a Skin Inpaint Mask

To prevent the AI from altering critical, identity-defining regions like the eyes, nostrils, lips, and hair, we isolate the skin using a mask.

For developers automating this step, you can use a face-parsing model (like BiSeNet or Mediapipe) to programmatically generate a mask that selects only the cheek, forehead, nose bridge, and chin areas.

# Conceptual Python snippet for skin masking using Mediapipe
import cv2
import mediapipe as mp

mp_face_geometry = mp.solutions.face_geometry
# [Load your image and extract the skin coordinates to create a binary mask]
# Save the mask where white (255) represents skin and black (0) represents protected areas (eyes, mouth, hair).

Step 3: Low-Denoising Image-to-Image Pass

Now, pass the pre-cleaned image and the skin mask into your image-to-image AI pipeline (such as Stable Diffusion Inpainting).

Because we have already neutralized the high-contrast blemishes, we can set the parameters defensively:

Denoising Strength: Set this between 0.15 and 0.25. This is high enough to let the generator synthesize micro-pore textures, but low enough that the structural geometry remains completely unchanged.
Prompt: Use a descriptive, quality-focused prompt that guides the texture synthesis without adding heavy stylistic elements.
- Example prompt: extreme close up portrait, highly detailed skin texture, pores, natural lighting, soft focus background, realistic photograph
- Negative prompt: painting, drawing, illustration, airbrushed, plastic skin, blurry, smooth, noise, artifacts

4. Programmatic Implementation Example

For developers looking to integrate this workflow into their applications, here is how you might structure an API payload for an automated pipeline using a Stable Diffusion WebUI or similar backend:

{
  "init_images": ["base64_encoded_pre_cleaned_image_here"],
  "mask": "base64_encoded_skin_mask_here",
  "mask_blur": 4,
  "inpainting_fill": 1, 
  "inpaint_full_res": true,
  "inpaint_full_res_padding": 32,
  "inpainting_mask_invert": 0,
  "prompt": "high fidelity raw photo skin texture, fine pores, subtle skin details, soft natural lighting",
  "negative_prompt": "smooth, glossy, plastic, oil painting, airbrushed, drawing, disfigured",
  "steps": 25,
  "sampler_name": "Euler a",
  "cfg_scale": 6.5,
  "denoising_strength": 0.20,
  "width": 512,
  "height": 512
}

By keeping the denoising_strength at 0.20 and targeting only the skin via the mask, the AI acts as a sophisticated texture synthesizer. It seamlessly fills in any flat or blurry patches left behind by the initial blemish remover pass, replacing them with plausible, natural skin cells and pores that match the scene's lighting.

5. Summary of Best Practices

To consistently achieve natural results with this workflow, keep these tips in mind:

Don't skip the prep work: Trying to save time by skipping the initial blemish remover step will force you to raise the AI's denoising strength, which often leads to a loss of the subject's unique facial structure.
Mind the moles and scars: Distinctive marks like moles, dimples, or character-defining scars should be protected. If your mask includes them, the AI might smooth them out. Keep these areas black on your inpaint mask to preserve them.
Keep the resolution matched: Ensure your input image resolution is high enough for the AI model to generate convincing micro-textures. Standard SD1.5 or SDXL models perform best when processing tiles or masked regions close to their native training resolutions (512px to 1024px).

By pairing the structural predictability of traditional pixel-healing algorithms with the texture-generation capabilities of generative AI, we can build pipelines that respect both efficiency and artistic realism.

DEV Community