DEV Community

eden zhang
eden zhang

Posted on

Understanding Perceptual Hashing: How to Identify Similar Images with a 64-bit Fingerprint

Why Traditional Hashing Fails for Images
When you need to deduplicate thousands of images scraped from a website, your first instinct might be to use MD5 or SHA-256. After all, these cryptographic hashes are the standard way to detect identical files.
But here's the problem: two images that look identical to the human eye can have completely different MD5 hashes. A slight compression, a re-encode, a cropped border, or an added watermark changes every byte of the file — even though the visual content is essentially the same.
What you need is a perceptual hash — a fingerprint that captures the visual essence of an image, not its exact byte sequence.
How pHash Works: The DCT Approach
The most popular perceptual hashing algorithm is pHash, which stands for "perceptual hash." It uses the Discrete Cosine Transform (DCT) — the same mathematical foundation behind JPEG compression.
Here's the step-by-step pipeline:
Step 1: Image Preprocessing
Resize the image to 32×32 pixels
Convert to grayscale
This reduces noise and normalizes dimensions
Step 2: Apply DCT
Compute the 2D Discrete Cosine Transform on the 32×32 image
DCT transforms the image from the spatial domain into the frequency domain
The top-left 8×8 coefficients represent the lowest frequencies — the most visually significant patterns
Step 3: Extract Low-Frequency Coefficients
Keep only the top-left 8×8 block (64 coefficients)
These 64 values capture the coarse structure of the image, ignoring fine details and noise
Step 4: Compute the Median
Calculate the median value of these 64 coefficients
The median is used as the threshold because it's robust to outliers
Step 5: Generate the 64-bit Hash
For each of the 64 coefficients: set the bit to 1 if the coefficient is above the median, 0 otherwise
The result is a 64-bit (16 hex character) fingerprint
Why This Works: Robustness to Transformations
The magic of pHash lies in its robustness:
Compression/re-encoding: DCT low frequencies are preserved even under heavy compression
Resizing: The 32×32 preprocessing normalizes all images to the same dimensions
Brightness/contrast changes: The median threshold adapts to overall brightness shifts
Minor crops: Low-frequency patterns remain largely intact
Measuring Similarity: Hamming Distance
To compare two pHash fingerprints, you compute the Hamming distance — the number of bits that differ between the two 64-bit hashes.
Distance 0: Identical hash (very likely the same image)
Distance 1-5: Very similar images (different compression, minor edits)
Distance 6-10: Possibly related images
Distance >10: Different images
In practice, a threshold of 5-10 works well for most deduplication tasks.
Implementation Challenges
While the pHash algorithm sounds straightforward, implementing it at scale presents real challenges:
Performance: Computing DCT for thousands of images can be slow. Optimized implementations use precomputed cosine tables and SIMD instructions.
Memory: Storing 64-bit hashes for 10,000 images is trivial (80KB), but computing pairwise Hamming distances is O(n²) — 50 million comparisons for 10,000 images.
False positives/negatives: The threshold is a tradeoff. Too low and you miss duplicates; too high and you flag distinct images as duplicates.
Text-heavy images: pHash works best for photographs. Images with lots of text or sharp edges may produce less reliable hashes.
OmniPic: pHash Deduplication Built In
Implementing all of this — image preprocessing, DCT computation, Hamming distance matching, threshold tuning — into a polished tool is a significant engineering effort.
OmniPic AI Studio Pro packages pHash visual deduplication into a ready-to-use browser extension. When you crawl images from any website, OmniPic automatically:
Computes pHash fingerprints for every detected image
Identifies and groups visually similar duplicates
Lets you filter out duplicates with one click
Works even when images have been cropped, compressed, or watermarked
But pHash deduplication is just one piece of OmniPic's full image workflow:
Full-Site Image Crawling & Auto-Classification
Automatically crawl all images from the current page (or entire sites), auto-classified by resolution into 4K Ultra HD, 1080P HD, medium-large, and thumbnail categories.
CDN Original Image Restoration
Automatically recognize URL patterns from Alibaba Cloud OSS, Tencent COS, Qiniu, and other CDNs to restore original high-resolution image addresses — so you download the source quality, not compressed thumbnails.
On-Device AI Reverse Image Search
Powered by MobileNet 1024-dimensional feature extraction with WebGPU acceleration, OmniPic finds visually similar images among all crawled images in under 50ms — entirely on your device, with no data leaving your browser.
Eagle/Billfish Direct Integration
Push selected images directly to your Eagle or Billfish local design asset library, or stream-pack them into a ZIP file (supports 2000+ images without freezing your browser).
100% Local Processing, Privacy First
All image processing, hashing, AI inference, and data storage happen locally in your browser. Your images never touch any server.
Get Started
OmniPic AI Studio Pro is available on all three major browser extension stores:
Chrome: Chrome Web Store
Edge: Microsoft Edge Add-ons
Firefox: Firefox Add-ons
Whether you're a designer collecting inspiration, an e-commerce operator scraping product images, or a content creator building image libraries, OmniPic turns the complex mathematics of perceptual hashing and AI feature extraction into one-click simplicity.

Top comments (0)