How to Tame Your AI Pixel Art

Eugeniy Smirnov — Wed, 30 Jul 2025 09:01:53 +0000

Over the past couple of years, generative AI has become a kind of magic brush for everything — concept art, icons, illustrations, covers, avatars, even sprites. Especially pixel art.
You just type something like "Pixel art goose with goggles in the style of SNES" into Midjourney, Stable Diffusion, DALL·E, or Image-1 — and bam, you've got yourself a glorious pixelated goose in under 10 seconds.

But if you’ve ever tried putting that goose into an actual game —
you already know the pain.

So I decided to dig deeper - and build an open-source tool that fixes AI-generated pixel art and turns it into clean, game-ready, pixel-perfect sprites.

Before we dive in, there’s a link to the tool and source code waiting for you at the end of the article — but first, let’s talk theory. Just a bit. I promise it’s useful.

Introduction: Why AI Still Doesn’t Get Pixel Art

Let’s get one thing straight — AI has no idea what a pixel is.
Modern models like Stable Diffusion don’t work with a pixel grid the way we do. They operate in a weird fuzzy space called latent space, where images start as pure noise — literally a cloud of randomized fog — and gradually get shaped into something that looks like an image.

It’s a beautiful process…
But it’s not discrete. There’s no concept of hard edges, clean pixels, or fixed palettes. Everything is smooth, continuous, and a little dreamy.

Let’s take a look at what that means for a typical piece of AI-generated “pixel art”:

First off, the pixel grid is all over the place — nothing aligns the way it should.

Second, even with a fixed grid, most “pixels” don’t land perfectly inside it — they bleed over, blur at the edges, or just float slightly off.

Third, if we extract the actual color palette from the image, it looks something like this:

So if we perform a naive downscale — say, using nearest neighbor — we end up with something like this:

So how do we fix this?

To turn AI-generated art into true, pixel-perfect assets, we need a smarter pipeline. One that can automatically:

Detect the pixel size
Find the right grid alignment
Extract a clean, limited palette
Downscale without losing important details
Clean up noise and artifacts in the final image

Step 1: Detecting Pseudo-Pixel Scale

To start, we need to figure out the size of the “fake pixels” in the AI-generated image.
I use an edge-aware detection method based on Sobel gradients + tile-based voting.

Step 1.1: Selecting informative tiles

The image is split into a 3×3 grid, and I pick the tiles with the highest variance — these are the ones most likely to contain meaningful edges, not flat backgrounds or empty space.

Step 1.2: Finding edges using the Sobel filter

The Sobel filter helps us detect areas with sharp color transitions — in other words, the borders between fake pixels.

We apply it in two directions:

Sobel X for vertical edges
Sobel Y for horizontal edges

This gives us two 2D maps of “sharpness” — bright lines show where the image pretends to be pixelated.

Step 1.3: Generating the edge profile

Next, we convert the 2D edge maps into simple 1D profiles.
For vertical edges (Sobel X), we sum up brightness column by column.
For horizontal edges (Sobel Y), we sum row by row.

The result is a pair of clean graphs — where peaks in the curve hint at potential pixel boundaries.

Step 1.4: Voting for the pixel scale

Now that we have the peaks, we look at the distances between them.
The idea is simple: if a fake pixel grid exists, it’ll leave a rhythmic pattern — repeating every N pixels.

So we collect all the distances between neighboring peaks and build a histogram of those distances.

In most test images, common step sizes like 8, 12, 16, or 24 pixels show up clearly.
(43px just happens to make a nice demo case 🙂)

To make things more visual, we highlight the strongest color transitions across the entire image — in other words, the borders between pseudo-pixels.

This heatmap helps confirm that the structure is regular, and that the detected scale actually matches the “grid” the AI was trying to fake.

Warm zones = strong edges where the neural net likely “drew” pixel boundaries.
The more regular these zones look, the more confident we can be in our scale detection.

Step 2: Grid Alignment & Smart Cropping

So, we’ve figured out the pixel scale — in our example, that’s 43×43 pixels.
But that’s only half the job.

Knowing the size of the grid isn’t enough — we also need to figure out where the grid starts.
In other words: how should we align it so that the grid lines actually match the structure of the image?

Here’s how we do it:

Convert the image to grayscale — we only care about structure, not color.
Use the Sobel filter again to detect sharp edges — these are likely to be fake pixel borders.
Build 1D edge profiles — summing edge intensity row-by-row and column-by-column.
Loop through all possible offsets from 0 to scale - 1, and for each one, check how well the grid lines up with the edge peaks.
Pick the offset (x, y) where the grid best matches the image’s internal structure.

function findOptimalCrop(grayMat, scale, cv) {
  const sobelX = new cv.Mat();
  const sobelY = new cv.Mat();

  try {
    cv.Sobel(grayMat, sobelX, cv.CV_32F, 1, 0, 3);
    cv.Sobel(grayMat, sobelY, cv.CV_32F, 0, 1, 3);

    const profileX = new Float32Array(grayMat.cols).fill(0);
    const profileY = new Float32Array(grayMat.rows).fill(0);
    const dataX = sobelX.data32F;
    const dataY = sobelY.data32F;

    for (let y = 0; y < grayMat.rows; y++) {
      for (let x = 0; x < grayMat.cols; x++) {
        const idx = y * grayMat.cols + x;
        profileX[x] += Math.abs(dataX[idx]);
        profileY[y] += Math.abs(dataY[idx]);
      }
    }

    const findBestOffset = (profile, s) => {
      let bestOffset = 0, maxScore = -1;
      for (let offset = 0; offset < s; offset++) {
        let currentScore = 0;
        for (let i = offset; i < profile.length; i += s) {
          currentScore += profile[i] || 0;
        }
        if (currentScore > maxScore) {
          maxScore = currentScore;
          bestOffset = offset;
        }
      }
      return bestOffset;
    };

    const bestDx = findBestOffset(profileX, scale);
    const bestDy = findBestOffset(profileY, scale);
    return { x: bestDx, y: bestDy };

  } finally {
    sobelX.delete();
    sobelY.delete();
  }
}

As a result, we now know both the size and the position of each pseudo-pixel.
Which means we can confidently draw a grid that matches the visual structure of the image.

At this point, we crop the image so that its dimensions are divisible by 43 —
ensuring that every grid block becomes a clean, well-defined cell we can safely analyze, downscale, and process without distortion.

Step 3: Building the Color Palette

Great pixel art isn’t just about the grid — it’s also about the palette.

Take retro consoles:

The NES had 54 visible colors (out of 64 total), and only 4 colors per tile
The SNES allowed more, but still operated with strict palette limits

In short, classic pixel art was always as much about restraint as it was about resolution.

For us, palette limitation solves multiple problems at once:

Reduces noise — eliminates soft gradients, anti-aliasing, and random color speckles
Brings that retro feel — the image looks clean and deliberate, as if it were actually drawn for the GBA or Mega Drive
Makes it easier to match your sprite with the rest of your game assets, especially if you're using a shared palette (like one from lospec.com)

To build the palette, I use Wu quantization, via the excellent image-q library.

Quantization is the process of collapsing similar colors into a smaller, fixed set.

For example, if an AI-generated image contains 1500 shades of green, quantization will reduce that to maybe 4 or 8 — and remap every pixel to the closest match.

Step 4: Downscaling with Dominant Color Voting

A simple nearest-neighbor downscale sometimes falls apart on AI-generated pixel art — edges get muddy, colors bleed, and the result just doesn’t feel right.

So I implemented a smarter method: block-wise color voting based on dominant color.

Here’s how it works:

We take each block (say, 43×43 pixels — one “pseudo-pixel”)
Count how many times each color appears inside the block
If a single color clearly dominates (over 5% more than the next), we assign it as the block’s final color
If there’s no clear winner — we fall back to a mean RGB blend to avoid visual gaps

Final Result

At the end of the pipeline, we get a clean, game-ready pixel art image — properly aligned, color-limited, and downscaled with intention.

There are also a couple of optional cleanup steps that help polish the result even further:

Morphological cleanup before downscaling — useful for removing stray pixels or minor noise left by the AI
Post-downscale quantization — sometimes new colors appear after blending or averaging, so we run one more palette cleanup pass to ensure the final image stays crisp and consistent

⚠️ Of course, not every image turns out perfect on the first try.

Some AI outputs are just too messy — or too painterly — for a fully automated cleanup. In those cases, you might need to tweak a few settings (like downscale method or target palette size), or give the final image a little pixel-editor love

(This tool won’t replace pixel artists — but it can definitely save them hours.)

More Examples

What’s next?

There’s still room to push this further.

One possible direction is integrating ideas from Depixelizing Pixel Art by Kopf & Lischinski (2011):
Convert the sprite into vector curves, clean them up, then re-rasterize with full control.

Other future improvements could include:

More advanced grid alignment algorithms
Alternative downscaling strategies with better structure preservation

If you’ve got ideas — or want to build something on top — the project is open-source and waiting for your forks.

Try it now

Interactive tool on Itch:
https://jenissimo.itch.io/unfaker

Source code on GitHub:
https://github.com/jenissimo/unfake.js
Pull requests are welcome! If you spot a bug, edge case, or just have an idea — feel free to open an issue or contribute.

If you enjoyed this article, feel free to follow me on X

Thanks for reading!

DEV Community: Eugeniy Smirnov