Most home-grown image scrapers fall apart the same way: they get CAPTCHA-walled after a few dozen requests, or they return a pile of irrelevant junk mixed in with the images you actually wanted. Universal Google Image Bulk Downloader Pro, an open-source project by hariompatel61, tackles both problems with a handful of deliberate architectural choices worth breaking down.
1. Undetected ChromeDriver + Persistent Sessions
Rather than relying on a headless request library, the tool drives a real, undetected instance of Chrome via undetected-chromedriver. This matters because standard Selenium/ChromeDriver setups leave detectable fingerprints that anti-bot systems flag quickly.
The bigger trick, though, is session persistence. The first time you run the script, if Google throws a CAPTCHA at you, you get a short window to solve it manually. From that point on, the session cookies are stored in a local chrome_profile/ folder, so every subsequent run reuses that trusted session instead of starting from a blank, suspicious slate.
2. A Two-Tier Search Strategy
Instead of firing a single generic query at Google Images, the tool runs a layered search:
- Tier 1 (Premium First): It biases results toward known stock photography domains — Freepik, Unsplash, Pexels, Pixabay — which tend to produce cleaner, higher-quality, and more consistently licensed-adjacent imagery.
-
Tier 2 (Intelligent Fallback): If the premium tier doesn't return enough usable results, it drops back to general Google Images search, but appends negative filters (like
-recipe -video) defined per job inconfig.json. This is what keeps a "food dish" search from returning cooking-tutorial thumbnails instead of clean product shots.
3. Config-Driven, Multi-Job Processing
Rather than hardcoding search behavior, everything lives in config.json. Each entry in the jobs array is essentially an independent scraping task: its own source spreadsheet, its own column mappings, its own search_context string, its own fallback filters, and its own output folder. That means a single script run can process an entirely different search strategy for a clothing catalog than it does for a food menu — no code changes required, just config edits.
4. Smart Header Detection
Real-world spreadsheets are messy. The tool is built to tolerate that — parsing single-column lists, two-column files with names and categories, or files with custom headers — rather than requiring a rigid, pre-formatted template.
5. Resume-and-Skip Logic
Long scraping jobs are fragile; a dropped connection partway through a 1,000-item list used to mean starting over. This tool checks the output folder before each download and skips anything that already exists, so interrupted runs can simply be restarted without wasted bandwidth or duplicate work.
6. Built-In Rate Limiting
Perhaps the most understated but important design decision: the script bakes in randomized delays — roughly 1.5 to 3.5 seconds between actions, with longer page waits around 3.5 seconds — keeping the average request rate around 10–15 per minute. This isn't just about avoiding bans; it's a basic courtesy to the search infrastructure being used, and the README explicitly warns against stripping these delays out.
The Net Result
None of these ideas — undetected browser automation, tiered search, config-driven job queues, resume logic, rate limiting — are individually novel. What makes this project useful is that they're combined into a single, MIT-licensed, ready-to-run tool instead of being something you'd have to stitch together yourself.
Source code and full configuration reference: github.com/hariompatel61/universal-image-downloader
Top comments (0)