Building an AI Image Upscaler with Python + Real-ESRGAN

Low-resolution images are a fact of life — compressed thumbnails, old screenshots, or that video frame you desperately need to reuse. As a developer, you don't want to depend on a cloud API with monthly limits. You want something you control, that runs locally, and that you can script into a pipeline.

In this tutorial I'll show you how to build your own AI image upscaler with Python and Real-ESRGAN — a state-of-the-art super-resolution model that runs completely offline.

Why Real-ESRGAN?

Real-ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) is one of the best open-source upscaling models available:

• Runs locally — no API keys, no monthly limits, no privacy concerns
• Handles real-world images well — trained to fix compression artifacts, blur, and noise, not just clean synthetic images
• Multiple scale factors — 2x, 4x, and custom upscaling
• Active community — models are continuously improved, with specialized variants for faces and anime

Prerequisites

# Python 3.8+ recommended
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118 # GPU
# or CPU-only:
pip install torch torchvision

pip install basicsr facexlib gfpgan
pip install realesrgan

Note: If you're on a Mac (MPS) or CPU-only machine, the same code works — it'll just be slower. A 4K upscale of a 1MP image takes roughly 30-60s on CPU.

Step 1: The Core Upscaler

Here's a minimal, dependency-tight version that handles the common cases:

import argparse
import time
from pathlib import Path

import cv2
import torch
from realesrgan import RealESRGANer
from basicsr.archs.rrdbnet_arch import RRDBNet


def build_upscaler(scale=4, gpu=True):
    model = RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32, scale=scale)
    weights = "https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth"
    device = torch.device("cuda" if gpu and torch.cuda.is_available() else "cpu")
    print(f"[INFO] Using device: {device}")
    upscaler = RealESRGANer(scale=scale, model_path=weights, model=model, tile=0, tile_pad=10, pre_pad=0, half=False, device=device)
    return upscaler


def upscale_image(upscaler, input_path, output_path):
    img = cv2.imread(input_path, cv2.IMREAD_COLOR)
    if img is None:
        raise FileNotFoundError(f"Could not read image: {input_path}")
    output, _ = upscaler.enhance(img, outscale=4)
    cv2.imwrite(output_path, output)
    print(f"[OK] {input_path} -> {output_path}")


if __name__ == "__main__":
    import argparse
    parser = argparse.ArgumentParser(description="AI image upscaler")
    parser.add_argument("input", type=str)
    parser.add_argument("--output", default="output.png")
    parser.add_argument("--scale", type=int, default=4, choices=[2, 4])
    parser.add_argument("--cpu", action="store_true")
    args = parser.parse_args()
    upscaler = build_upscaler(scale=args.scale, gpu=not args.cpu)
    upscale_image(upscaler, args.input, args.output)

Run it:

python upscaler.py input.jpg --output result.png --scale 4

Step 2: Batch Processing a Directory

Single images are nice, but real workflows need batches. Let's add directory support:

def upscale_directory(upscaler, in_dir, out_dir, extensions=(".jpg", ".jpeg", ".png", ".webp")):
    in_dir = Path(in_dir)
    out_dir = Path(out_dir)
    out_dir.mkdir(parents=True, exist_ok=True)
    images = [p for p in in_dir.iterdir() if p.suffix.lower() in extensions]
    print(f"[INFO] Found {len(images)} images to process")
    for img_path in images:
        out_path = out_dir / f"{img_path.stem}_x4.png"
        try:
            upscale_image(upscaler, str(img_path), str(out_path))
        except Exception as e:
            print(f"[ERROR] Failed {img_path.name}: {e}")

For large directories, add tiling to avoid running out of GPU memory:

upscaler = RealESRGANer(scale=4, model_path=weights, model=model, tile=400, tile_pad=10, pre_pad=0, half=False, device=device)

Tiling trades a tiny bit of seam quality for massive memory savings — essential when upscaling 8K+ images or running on a laptop GPU.

Step 3: Face Enhancement (GFPGAN)

Real-ESRGAN alone struggles with faces — they can come out waxy or distorted. The GFPGAN model fixes that:

from gfpgan import GFPGANer

def build_face_upscaler(device="cpu"):
    face_enhancer = GFPGANer(
        model_path="https://github.com/TencentARC/GFPGAN/releases/download/v1.3.0/GFPGANv1.3.pth",
        upscale=2,
        arch="clean",
        channel_multiplier=2,
        bg_upsampler=None,
        device=device,
    )
    return face_enhancer

This is a game-changer for old family photos or low-res profile pictures.

Step 4: A Tiny Web API (FastAPI)

Want to expose your upscaler as a service? Here's a minimal FastAPI endpoint:

from fastapi import FastAPI, File, UploadFile
from fastapi.responses import Response
import numpy as np
import cv2

app = FastAPI()
upscaler = build_upscaler(scale=4, gpu=False)

@app.post("/upscale")
async def upscale(file: UploadFile = File(...)):
    data = await file.read()
    img = cv2.imdecode(np.frombuffer(data, np.uint8), cv2.IMREAD_COLOR)
    output, _ = upscaler.enhance(img, outscale=4)
    ok, encoded = cv2.imencode(".png", output)
    return Response(content=encoded.tobytes(), media_type="image/png")

Start it with:

uvicorn api:app --host 0.0.0.0 --port 8000

And test:

curl -X POST -F "file=@photo.jpg" http://localhost:8000/upscale -o result.png

Benchmarks (Quick Numbers)

Rough timings on a mid-range RTX 3060 (12GB):

Input size	Scale	Time	VRAM
512x512	4x	~1.2s	~1.5GB
1024x1024	4x	~4s	~4GB
2048x2048	4x (tile=400)	~18s	~4.5GB
CPU (M1)	4x	30-60s	N/A

Common Pitfalls

1. Black output images — usually a dtype/range issue. Make sure input is uint8 and cv2.imwrite gets a valid array. If you used half=True and get NaNs, your GPU may not support FP16 well — set half=False.

2. Out of memory on big images — use tile=400 and tile_pad=10. Never upscale an 8K image without tiling on a consumer GPU.

3. CUDA errors on fresh installs — install PyTorch with the correct CUDA index URL (cu118, cu121, etc.) matching your driver.

4. Models won't auto-download — set model_path to a local file or pre-download manually with wget.

Full Repository Structure

upscaler/
├── upscaler.py          # core + CLI
├── batch.py             # directory processing
├── face.py              # GFPGAN face enhancement
├── api.py               # FastAPI service
├── requirements.txt
└── README.md

Wrapping Up

You now have a fully local, scriptable AI image upscaler. No API keys, no per-image costs, no data leaving your machine. It handles single images, batches, faces, and can even run as a small web service.

Start small — upscale a few test images with --scale 2 first to gauge quality, then push to 4x for the keepers. The model is surprisingly good at making compressed JPEGs look clean again.

If you found this useful, check out the original article on AIXHDD for more AI tooling breakdowns and creative workflows.