Batch image processing in Python: threads vs processes with Pillow

For a Pillow batch that opens, resizes and saves images, use a thread pool. In my run, 24 large photos took 6.28 s one at a time, 1.33 s on 8 threads and 1.91 s on 8 processes. Threads win because the heavy work runs in Pillow's C code, which let 8 threads run side by side here. Once your own Python code touches the pixels, the GIL is back, and processes win.

Why would threads help at all with the GIL?

The GIL stops two threads from running Python bytecode at once. It does not stop C code that has released it. Decoding a JPEG, resizing with LANCZOS and encoding a JPEG all happen inside Pillow's C extension. If those calls release the GIL, threads run in parallel. The way to know is to measure it, so I did.

The batch script

The task is the one most batch jobs do: open a photo, apply its EXIF orientation, resize to 1600 px wide, save a JPEG at quality 75.

import os, sys, time, statistics
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
from pathlib import Path
from PIL import Image, ImageOps

SRC = Path(sys.argv[1])
OUT = Path("out"); OUT.mkdir(exist_ok=True)


def squeeze(path: Path) -> int:
    with Image.open(path) as im:
        im = ImageOps.exif_transpose(im)
        w = 1600
        im = im.resize((w, round(im.height * w / im.width)), Image.LANCZOS)
        dest = OUT / path.name
        im.save(dest, "JPEG", quality=75, optimize=True)
    return dest.stat().st_size


def run(kind, workers, files):
    t0 = time.perf_counter()
    if kind == "serial":
        sizes = [squeeze(f) for f in files]
    else:
        Pool = ThreadPoolExecutor if kind == "threads" else ProcessPoolExecutor
        with Pool(max_workers=workers) as pool:
            sizes = list(pool.map(squeeze, files))
    return time.perf_counter() - t0, sum(sizes)


if __name__ == "__main__":
    files = sorted(SRC.glob("*.jpg"))
    for kind, workers in [("serial", 1), ("threads", 4), ("threads", 8),
                          ("processes", 4), ("processes", 8)]:
        times = []
        for _ in range(5):
            secs, total = run(kind, workers, files)
            times.append(secs)
        print(f"{kind:9} workers={workers}  median {statistics.median(times):.2f}s  "
              f"runs {' '.join(f'{t:.2f}' for t in times)}  out {total/1000:.1f} KB")

concurrent.futures gives both pools the same map() call, so the comparison changes one class name and nothing else. The if __name__ == "__main__": guard is required for the process pool on macOS: since Python 3.8 the default start method there is spawn (multiprocessing docs), which re-imports your script in every worker.

What did it print?

The input was the site's 4964 × 2782 camera photo, copied 24 times.

serial    workers=1  median 6.28s  runs 6.37 6.25 6.28 6.41 6.26  out 2849.2 KB
threads   workers=4  median 1.71s  runs 1.70 1.72 1.71 1.70 1.71  out 2849.2 KB
threads   workers=8  median 1.33s  runs 1.26 1.33 1.33 1.30 1.39  out 2849.2 KB
processes workers=4  median 1.96s  runs 1.96 2.14 1.85 1.85 2.06  out 2849.2 KB
processes workers=8  median 1.91s  runs 2.20 1.91 1.73 1.75 2.28  out 2849.2 KB

Pool	Workers	Median	Speed-up
none (serial)	1	6.28 s	1.0×
threads	4	1.71 s	3.7×
threads	8	1.33 s	4.7×
processes	4	1.96 s	3.2×
processes	8	1.91 s	3.3×

Threads scaled from 4 to 8 workers. Processes barely moved: going from 4 to 8 saved 0.05 s. Each process pays to start, import Pillow, and send its result back through a pipe, and with only 24 files those costs are a large share of the run. The output was 2,849.2 KB every time, so all five ran the same work.

When do processes win?

When your code, not Pillow's, does the work. I wrote a second task that decodes each photo, halves it with reduce(2), and then sums the pixel values in a plain Python loop:

def mean_luma_python(path: Path) -> float:
    # Pillow decodes and shrinks in C; the loop below is pure Python and holds the GIL.
    with Image.open(path) as im:
        small = im.convert("L").reduce(2)
    px = small.tobytes()
    total = 0
    for v in px:
        total += v
    return total / len(px)

serial    workers=1  median 5.11s  runs 5.11 5.09 5.07 5.14 5.29
threads   workers=8  median 2.80s  runs 2.67 2.82 2.80 2.80 2.67
processes workers=8  median 1.33s  runs 1.41 1.33 1.47 1.25 1.32

Threads still helped (the decode is C), but the Python loop serialised them on the GIL. Processes each have their own interpreter and their own GIL, so they ran the loop in parallel and finished in under half the thread time.

Note: the rule is not "images need processes". It is "whoever runs the hot loop decides". Pillow's own decode, resize and encode ran in parallel on threads here. A per-pixel loop in Python, or a library that keeps the GIL, needs processes. With NumPy, most array maths also runs in C, so measure before you reach for a process pool.

Which one should you use?

Start with ThreadPoolExecutor(max_workers=os.cpu_count()) for open, resize, convert and save. It shares memory, starts instantly and needs no __main__ guard. Switch to ProcessPoolExecutor when a profile shows time in your own Python code, and give each worker a chunk of files with map(..., chunksize=n) so the start-up cost spreads over more work.

The same split exists in Node, where sharp runs pipelines on a fixed thread pool; the PNG compressor and the rest of the site run the engine in queue workers for the same reason: encoding is CPU work that belongs off the request path.

How I measured

• Machine: Apple M1 Pro (8 cores: 6 performance, 2 efficiency), macOS, 2026-10-17.
• Python 3.9.6 in a fresh venv, Pillow 11.3.0 (libjpeg-turbo 3.1.1 and zlib-ng, as PIL.features reports them).
• Input: image-compressor-sample-camera-photo.jpeg from the site (4964 × 2782, 9,804,300 bytes), copied 24 times.
• Each setting ran 5 times; the outputs show every run and the median.
• A first pass ran while other jobs loaded the machine and scattered from 6.42 s to 15.88 s for the serial case; I discarded it and reran once the load average had dropped.

What I did not test

Python 3.13's free-threaded build, Linux with the fork start method, and Pillow-SIMD.

---

I build Blinkimg, an image compressor and converter with a developer API. I wrote this article with help from an AI assistant (Claude), which ran the code on my machine; every number in it comes from those runs.