Sahadat Husain

Posted on Oct 22

Detect image formats from magic bytes in Node and Python

#node #python #security #webdev

Read the first bytes of the file and compare them with each format's signature: FF D8 FF is JPEG, 89 50 4E 47 0D 0A 1A 0A is PNG, RIFF plus WEBP at byte 8 is WebP, and an ftyp box with an avif or heic brand is AVIF or HEIC. A 20-line function in Node or Python does it from 64 bytes. On my 12 test files it named every format correctly; the file extension was wrong on 4 of them.

Here is the sniffer in both languages, and what sharp, Pillow and Python's imghdr said about the same files.

Why not trust the file extension or the Content-Type?

Both come from the client. A user renames screenshot.png to .jpg, a phone uploads HEIC as image.jpeg, and an attacker names an SVG avatar.jpg. Your decoder does not care what the name says. It reads the bytes, and some decoders accept more formats than you planned for.

That last case is the one to worry about. I wrote a 64 × 64 SVG to a file called svg-as.jpg and handed it to sharp:

files/svg-as.jpg         metadata=svg            decode=ok

sharp detected SVG from the content and rendered it. If your upload route only meant to accept photos, it just ran an XML renderer on user input. Sniff first, and refuse anything that is not on your list.

How do you sniff image magic bytes in Node?

Read a small head of the file, never the whole thing, and check each signature:

// sniff.mjs: node sniff.mjs files/*
import { open } from 'node:fs/promises';

const ascii = (b, start, end) => b.toString('latin1', start, end);

export function sniff(b) {
  if (b.length >= 3 && b[0] === 0xff && b[1] === 0xd8 && b[2] === 0xff) return 'jpeg';
  if (b.length >= 8 && b.subarray(0, 8).equals(Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]))) return 'png';
  if (ascii(b, 0, 6) === 'GIF87a' || ascii(b, 0, 6) === 'GIF89a') return 'gif';
  if (ascii(b, 0, 4) === 'RIFF' && ascii(b, 8, 12) === 'WEBP') return 'webp';
  if (ascii(b, 4, 8) === 'ftyp') {
    // ISO BMFF: a major brand, a minor version, then compatible brands, 4 bytes each.
    const size = Math.min(b.readUInt32BE(0), b.length);
    const brands = [ascii(b, 8, 12)];
    for (let i = 16; i + 4 <= size; i += 4) brands.push(ascii(b, i, i + 4));
    if (brands.some((x) => x === 'avif' || x === 'avis')) return 'avif';
    if (brands.some((x) => ['heic', 'heix', 'heim', 'heis', 'mif1'].includes(x))) return 'heic';
  }
  return null;
}

async function head(path, n = 64) {
  const fh = await open(path, 'r');
  try {
    const { buffer, bytesRead } = await fh.read(Buffer.alloc(n), 0, n, 0);
    return buffer.subarray(0, bytesRead);
  } finally {
    await fh.close();
  }
}

if (import.meta.url === `file://${process.argv[1]}`) {
  for (const path of process.argv.slice(2)) {
    const ext = path.split('.').pop();
    console.log(`${path.padEnd(24)} ext=${ext.padEnd(5)} sniffed=${sniff(await head(path))}`);
  }
}

The signatures come from the specs: the 8-byte PNG signature in RFC 2083, and the 12-byte RIFF header in Google's WebP container spec, which says a WebP file must begin with RIFF and the FourCC WEBP.

What it printed:

files/graphic.gif        ext=gif   sniffed=gif
files/not-an-image.jpg   ext=jpg   sniffed=null
files/photo.avif         ext=avif  sniffed=avif
files/photo.heic         ext=heic  sniffed=heic
files/photo.jpg          ext=jpg   sniffed=jpeg
files/photo.png          ext=png   sniffed=png
files/photo.webp         ext=webp  sniffed=webp
files/renamed-avif.png   ext=png   sniffed=avif
files/renamed-png.jpg    ext=jpg   sniffed=png
files/svg-as.jpg         ext=jpg   sniffed=null
files/transparent.png    ext=png   sniffed=png
files/truncated.jpg      ext=jpg   sniffed=jpeg

Why does AVIF need the brand list, not just the first brand?

AVIF and HEIC share one container, ISO BMFF. Both start with a box size and ftyp. Only the brands tell them apart, and the major brand alone is not enough. Our AVIF's header lists avif first, then mif1, avif and miaf as compatible brands. Another encoder can put mif1 first and avif further down. Check the whole list, and check avif before the generic mif1.

How do you do the same in Python?

The same logic in plain bytes slicing:

# sniff.py: python sniff.py files/*
import sys

PNG = b'\x89PNG\r\n\x1a\n'

def sniff(b: bytes):
    if b[:3] == b'\xff\xd8\xff':
        return 'jpeg'
    if b[:8] == PNG:
        return 'png'
    if b[:6] in (b'GIF87a', b'GIF89a'):
        return 'gif'
    if b[:4] == b'RIFF' and b[8:12] == b'WEBP':
        return 'webp'
    if b[4:8] == b'ftyp':
        size = min(int.from_bytes(b[0:4], 'big'), len(b))
        brands = [b[8:12]] + [b[i:i + 4] for i in range(16, size - 3, 4)]
        if b'avif' in brands or b'avis' in brands:
            return 'avif'
        if any(x in brands for x in (b'heic', b'heix', b'heim', b'heis', b'mif1')):
            return 'heic'
    return None

if __name__ == '__main__':
    for path in sys.argv[1:]:
        with open(path, 'rb') as f:
            head = f.read(64)
        print(f"{path:<24} sniffed={sniff(head)}")

It printed the same 12 answers as the Node version.

What do sharp, Pillow and imghdr say about the same files?

File	Sniffer	`sharp.metadata()`	Pillow `Image.open`	`imghdr.what`
photo.jpg	jpeg	jpeg	JPEG	None
photo.avif	avif	heif/av1	AVIF	None
photo.heic	heic	heif/hevc	not identified	None
renamed-png.jpg	png	png	PNG	png
svg-as.jpg	null	svg, decodes	not identified	None
not-an-image.jpg	null	error	not identified	None
truncated.jpg	jpeg	jpeg, decode fails	JPEG, load fails	None

Three results stand out:

imghdr missed a normal JPEG. Its JPEG test looks for the text JFIF or Exif at bytes 6 to 10. Our JPEG starts FF D8 FF DB with no APP0 or APP1 segment, so it returned None. The module is also gone: the Python docs deprecated it in 3.11 for removal in 3.13 (PEP 594).
sharp reports AVIF and HEIC both as heif. Read compression (av1 or hevc) to tell them apart. sharp read the HEIC's metadata here but failed to decode it.
Pillow 11.3.0 opened the AVIF with no plugin, and could not identify the HEIC.

Is sniffing slower than asking the decoder?

No. On the 9.8 MB camera JPEG, median of 5 runs each:

sniff 64 bytes    0.11 ms
sharp.metadata()  0.40 ms
full decode       123.53 ms

Sniffing costs less than a millisecond, so run it before anything else touches the file.

Note: Magic bytes tell you what a file claims to be, not that it is valid. truncated.jpg is the first 2,000 bytes of a real JPEG: it passed the sniffer and sharp.metadata(), then failed at decode with "premature end of JPEG image". Sniff to reject, then decode inside a try with pixel limits, and treat a decode failure as a bad upload.

Our engine sniffs every upload the same way before it decodes. If you want to see how a renamed or odd file is handled end to end, the image format guide covers what each of these formats is for.

How I measured

Files: photo.jpg, photo.png, photo.webp, photo.avif, photo.heic, transparent.png from packages/engine/tests/fixtures/; graphic.gif converted by sharp from graphic.png; two copies renamed; truncated.jpg (first 2,000 bytes of photo.jpg); two SVG text files named .jpg. Timing on apps/web/public/images/index/image-compressor-sample-camera-photo.jpeg.
Node 24.22.0 with sharp 0.35.4 (libvips 8.18.6). Python 3.9.6 with Pillow 11.3.0, in a venv.
Time: median of 5 runs, performance.now().
Apple M1 Pro, macOS, 2026-10-17.

What I did not test

TIFF, BMP, ICO, JPEG XL and animated AVIF (avis) files. I did not test the file-type npm package or python-magic.

I build Blinkimg, an image compressor and converter with a developer API. I wrote this article with help from an AI assistant (Claude), which ran the code on my machine; every number in it comes from those runs.

DEV Community